IDKCS005059
Sync Master & Slave Repository In A High Availability Data Mover Configuration WrittenMar 23, 2017

 
Attachments0
  

Product Categories
Data Mover

 Objective
The article covers the process to sync the Master & Slave repositories in an High-Availability Data Mover configuration. The process is covered in the Data Mover User Guide Section "Synchronizing the Master and Slave Repositories". The following JIRAs require Manual Synchronization of Master & Slave Repository:
  • JIRA # DM-18391 Documented KCS # KCS001601 Fixed DM 15.10 & Onwards.
  • JIRA # DM-19627 Documented KCS # KCS001489 Fixed DM 15.11.00 & Onwards.
  • JIRA # DM-21508 Documented KCS # KCS002939 Fixed DM 15.11.02 & Onwards.
Collectively, all JIRAs have been Fixed @ DM 15.11.02 & onwards. However, we strongly recommend Customer to upgrade to DM 15.11.05 or later as DM 15.11.05 is Stable than DM 15.11.02.
 
The KCS is a step by step practical demonstration of the steps to ensure Customer or Site team perform such activities as GSO will not perform the Billable Service as Fix Version is already available for Customer.
 
 

Environment/Conditions/Configuration
  • ​Data Mover (DM) - Any Release


Procedure
In an High-Availability Data Mover configuration, we have 01 Master Server & 01 Slave Server. We can have multiple Agent Only Server, yet synchronization is performed for Master & Slave Server only. The Database on Agent-Only Server isn't Up. Only the Master & Slave Server have an up & running database. For successful failover, the data across both Master & Slave Server should be the same. Owing to multiple factors (Network, Code Bug, Incorrect Configuration), the data across both Master & Slave DM Server may not be in sync. The article covers the process to sync the Master & Slave Server repositories by Archiving the Master Server repository & Restoring the same to the Slave Server repository.
 
In the example below, Master Server is MSMProd & Slave Server is MSMDa.
 
​01. Stop the Monitor Service:
msmprod:~ # /opt/teradata/client/15.11/datamover/failover/dmcluster stopmonitor
Stopping Monitoring Service on local and remote monitoring servers...
Stop Monitor completed successfully.
 
02. Ensure no DM Jobs are Running:
msmprod:~ # datamove list_jobs -status_mode R
Data Mover Command Line 15.11.05.00
status_mode overridden by command line: R
2017-03-23 11:54:14 - List Jobs mode
Command parameters:
- status_mode: R
About to connect to ActiveMQ at msmprod,msmda:61616
Connected to Daemon version 15.11.05.00
List jobs...
Job Name                                                               Start                End                  Status                 Priority
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
No Detailed Information
 
03. If DM Jobs are Running, Perform Stop-Cleanup of the DM Jobs.
 
04. Stop Sync Service On Slave DM Server:
msmda:~ # /opt/teradata/datamover/sync/15.11/dmsync stop
dmsync shutdown...
killing PID 31253
dmuser dmsync stopped
 
05. Stop Sync Service On Master DM Server:
msmprod:~ # /opt/teradata/datamover/sync/15.11/dmsync stop
dmsync shutdown...
killing PID 19294
dmuser dmsync stopped
 
06. Stop Daemon Service On Slave DM Server:
msmda:~ # /etc/init.d/dmdaemon stop
Shutting down Teradata DataMover dmdaemon with dmuser...                                                                                                     done
dmdaemon stopped
 
In A Correctly Configured HA Setup, Daemon Would Be Stopped Already On Slave DM Server, In Which Case Following Message Is Shown:
msmda:~ # /etc/init.d/dmdaemon stop
Shutting down Teradata DataMover dmdaemon with dmuser...                                                                                                     done
dmdaemon is not running
 
07. Ensure Daemon, Agent & TDActiveMQ Services are Up & Running On Master DM Server. If not, Start Them Via /etc/ini.t/dmdaemon|dmagent/tdactivemq start
 
08. Archive The Master DM Server Repository:
msmprod:~ # datamove backup_daemon
Data Mover Command Line 15.11.05.00
2017-03-23 11:34:49 - Backup Daemon mode
Command parameters:
About to connect to ActiveMQ at msmprod,msmda:61616
Connected to Daemon version 15.11.05.00
Backing up daemon...
- Daemon backup job is scheduled, backup files will be saved at location : /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50/. Daemon will be restarted
 
09. Check Archive Successful Completion (Severity 0 Is Good For Archive):
msmprod:~ # cd /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50
msmprod:/var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50 # ls -ltr
total 15776
-rwxr-x--- 1 dmuser users      669 Mar 23 11:34 launchScript_backup.bat
-rw-r----- 1 dmuser users        0 Mar 23 11:34 backup_server.log
-rw-r----- 1 dmuser users 15858905 Mar 23 11:34 db_archive
-rw-r----- 1 dmuser users   264543 Mar 23 11:34 backup_script.output
msmprod:/var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50 # tail -15 backup_script.output
03/23/2017 11:34:59 ---  Writing database trailer record.
03/23/2017 11:34:59 ---  Sending release lock request to DBS.
03/23/2017 11:34:59  "DATAMOVER" - LOCK RELEASED
03/23/2017 11:34:59 ---  Writing archive trailer record.
03/23/2017 11:34:59 ---  Closing archive file(s)
03/23/2017 11:34:59  DUMP COMPLETED
03/23/2017 11:34:59  STATEMENT COMPLETED
03/23/2017 11:34:59  
03/23/2017 11:34:59 ---  Statement does not require an access rights check.
03/23/2017 11:34:59 ---  Disconnecting all sessions.
03/23/2017 11:34:59  LOGGED OFF   6 SESSIONS
03/23/2017 11:34:59  STATEMENT COMPLETED
03/23/2017 11:34:59  
03/23/2017 11:34:59  
03/23/2017 11:34:59  ARCMAIN TERMINATED WITH SEVERITY 0
 
10. Copy The Backup Directory (Not Individual Files) From Active DM Server To Slave DM Server:
msmprod:~ # scp -r //var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50 root@msmda:/var/opt/teradata/datamover/daemon_backup/
Password:
backup_server.log                               100%    0     0.0KB/s   00:00    
launchScript_backup.bat                         100%  669     0.7KB/s   00:00    
backup_script.output                            100%  258KB 258.3KB/s   00:00    
db_archive                                      100%   15MB  15.1MB/s   00:01    
You have new mail in /var/mail/root
 
11. Stop The Daemon Service On Master DM Server (Stop Via /etc/init.d/dmdaemon stop or Alternative Approach Shared Below):
msmprod:/opt/teradata/datamover/daemon/15.11 # ./dmdaemon stop
Shutting down Teradata DataMover dmdaemon with dmuser...                                                                                                     done
dmdaemon stopped
 
12. Ensure Daemon, Agent & TDActiveMQ Services are Up & Running On Slave DM Server. If not, Start Them Via /etc/ini.t/dmdaemon|dmagent/tdactivemq start
 
13. Change Directory & File Permission & Ownership On Slave DM Server:
msmda: chown dmuser /var/opt/teradata/datamover/daemon_backup
msmda: chown /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50
msmda: chown dmuser /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50/*
msmda: chmod 755 /var/opt/teradata/datamover/daemon_backup
msmda: chmod 755 /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50
msmda: chmod 755 /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50/*
 
14. Run The Restore Job On Slave DM Server:
msmda:/var/opt/teradata/datamover/daemon_backup # datamove restore_daemon -backup_target_dir /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50
Data Mover Command Line 15.11.05.00
backup_target_dir overridden by command line: /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50
2017-03-23 12:19:09 - Restore Daemon mode
Command parameters:
- backup_target_dir: /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50
About to connect to ActiveMQ at msmda,msmprod:61616
Connected to Daemon version 15.11.05.00
Restoring daemon...
- Daemon restore job is scheduled, restore files are retrieved from : /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50/. Daemon will be restarted
 
15. Check Restore Successful Completion (Severity 4 Is Good For Restore As Restore Warns About Already Existing Objects):
msmda:/var/opt/teradata/datamover/daemon_backup # cd /var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50/
msmda:/var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50 # ls -ltr
total 18388
-rwxr-xr-x 1 dmuser root       669 Mar 23 11:50 launchScript_backup.bat
-rwxr-xr-x 1 dmuser root         0 Mar 23 11:50 backup_server.log
-rwxr-xr-x 1 dmuser root    264543 Mar 23 11:50 backup_script.output
-rwxr-xr-x 1 dmuser root  15858905 Mar 23 11:50 TEMP
-rwxr-x--- 1 dmuser users      809 Mar 23 12:19 launchScript_restore.bat
-rw-r----- 1 dmuser users      100 Mar 23 12:19 arcScript_restore.scr
-rw-r----- 1 dmuser users       61 Mar 23 12:19 restore_server.log
-rw-r----- 1 dmuser users  2595497 Mar 23 12:20 ARCLOG170323_121920_8566.rlg
-rw-r----- 1 dmuser users    54959 Mar 23 12:21 restore_script.output
msmda:/var/opt/teradata/datamover/daemon_backup/backup_2017-03-23_11.34.50 # tail -30 restore_script.output
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Clearing build flag for index 0.
03/23/2017 12:23:33 ---  Updating event row.
03/23/2017 12:23:33 ---  Sending End Transaction request
03/23/2017 12:23:34 ---  End data phase
03/23/2017 12:23:34 ---  Sending release lock request to DBS.
03/23/2017 12:23:34  "DATAMOVER" - LOCK RELEASED
03/23/2017 12:23:34   
03/23/2017 12:23:34 ---  Check if end of restart list.
03/23/2017 12:23:34 ---  Reached end of the restart log.
03/23/2017 12:23:34 ---  No Join/Hash Index data to restore.
03/23/2017 12:23:34 ---  No Stat Collection data to restore.
03/23/2017 12:23:34  STATEMENT COMPLETED
03/23/2017 12:23:34  
03/23/2017 12:23:34 ---  Statement does not require an access rights check.
03/23/2017 12:23:34 ---  Disconnecting all sessions.
03/23/2017 12:23:34  LOGGED OFF   6 SESSIONS
03/23/2017 12:23:34  STATEMENT COMPLETED
03/23/2017 12:23:34  
03/23/2017 12:23:34  
03/23/2017 12:23:34  ARCMAIN TERMINATED WITH SEVERITY 4
 
16. Remove all *.sql, *lastRead, *lastExecuted files From "/var/opt/teradata/datamover/logs" On Master & Slave DM Server.
 
17. Stop Daemon On Slave DM Server
msmda:/var/opt/teradata/datamover/logs # /etc/init.d/dmdaemon stop
Shutting down Teradata DataMover dmdaemon with dmuser...                                                                                                     done
dmdaemon stopped
 

Special Considerations

Additional Information