Hi Expert,
I have issue with our backup for BW production system. We are using Symantec netbackup.
The issue is sometime the backup is hanging. Please refer below logs:
bash-3.00# more log.082313
22:00:11.191 [21319] <2> logparams: -sb -rdbms db2 -S CS24SBCS1 -to 3600 -c SAP-PRD-BIS_db2_daily -s bwp_Full_Daily -clnt CSPM4BIS -FULL -kl 14 -b CSPM4BIS_137726640
7 -jobid 427592 -shm
22:00:11.263 [21319] <2> init_resilient_cache: [vnet_nbrntd.c:869] Initialize resilient cache. 0 0x0
22:00:11.263 [21319] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:808] the socket is 2 0x2
22:00:11.264 [21319] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:808] the socket is 0 0x0
22:00:11.392 [21319] <4> ncfLogConfiguration: Module: libncf, Release: 7.5.0.4 , Build: 08/30/2011 23:00:00 CDT (20110830)
22:00:11.392 [21319] <4> ncfLogConfiguration: System: SunOS, CSPM4BIS, 5.10, Generic_147440-15, sun4u, {5}
22:00:11.421 [21319] <4> NcfString::logLocale: Current code page: ICU: US-ASCII, Max. character width: 1, Substitution character: 0x1A [.], Unicode substitution char
acter: 0xfffd [. ]
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Running as user '0', primary group '0'
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable IS_NETBACKUP_DAEMON=YES
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable DISPLAY=localhost:11.0
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable HOME=/root
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable JAVA_HOME=/usr/j2se
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable LOGNAME=root
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable MAIL=/var/mail//root
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable PATH=/usr/j2se/bin:/usr/sbin:/usr/bin:/usr/bin
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable PWD=/root
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable SHELL=/sbin/sh
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable SHLVL=1
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable SSH_CLIENT=192.168.173.22 60302 22
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable SSH_CONNECTION=192.168.173.22 60302 172.21.33.31 22
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable SSH_TTY=/dev/pts/2
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable TERM=vt100
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable TZ=Asia/Kuala_Lumpur
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable USER=root
22:00:11.422 [21319] <4> Impersonate::logImpersonation: Environment variable _=/etc/init.d/netbackup
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_POLICY=SAP-PRD-BIS_db2_daily
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_SCHED=bwp_Full_Daily
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_SERVER=CS24SBCS1
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_CLIENT=CSPM4BIS
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_FULL=1
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_INCR=0
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_CINC=0
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_USER_INITIATED=0
22:00:11.422 [21319] <4> bphdb main: INF - setenv DB2_SCHEDULED=1
22:00:11.423 [21319] <4> bphdb sync_server: INF - BACKUP START 21319
22:00:11.433 [21319] <4> bphdb sync_server: INF - CONTINUE BACKUP message received
22:00:11.433 [21319] <16> bphdb get_filelist: set recv timeout error
22:00:11.438 [21319] <2> bphdb get_filelist: INF - Read filename: </usr/openv/netbackup/ext/db_ext/db2/scripts/db2_backup>
22:00:11.438 [21319] <16> bphdb get_filelist: set recv timeout error
22:00:11.438 [21319] <2> bphdb get_filelist: INF - Read filename: <CONTINUE>
22:00:11.438 [21319] <4> bphdb do_backup: INF - Processing /usr/openv/netbackup/ext/db_ext/db2/scripts/db2_backup
22:00:11.440 [21319] <4> bphdb do_backup: INF - Waiting for the child status
22:00:11.442 [21320] <4> bphdb do_backup: INF - Child executing /usr/openv/netbackup/ext/db_ext/db2/scripts/db2_backup
Db2diag.log
2013-08-23-22.00.33.453893+480 E3018620151A443 LEVEL: Info
PID : 20011 TID : 364 PROC : db2sysc 0
INSTANCE: db2bwp NODE : 000 DB : BWP
APPHDL : 0-36310 APPID: *LOCAL.db2bwp.130823140042
AUTHID : BWPADM
EDUID : 364 EDUNAME: db2agent (BWP) 0
FUNCTION: DB2 UDB, database utilities, sqlubSetupJobControl, probe:1508
MESSAGE : Starting an online db backup.
db2 list utilities show detail
ID = 10
Type = BACKUP
Database Name = BWP
Partition Number = 0
Description = online db
Start Time = 08/23/2013 22:00:33.453878
State = Executing
Invocation Type = User
Throttling:
Priority = Unthrottled
Progress Monitoring:
Estimated Percentage Complete = 0
Total Work = 1088530797606 bytes
Completed Work = 0 bytes
Start Time = 08/23/2013 22:00:33.453889
As you can see, the backup was triggered but no progress. the background workprocesses also fully occupied by report SAPMSSY2 starting around the time the backup was triggered(happen every time). i think these 2 issues are related. i wonder if the backup issue caused SAPMSSY2 to hang or vice versa. i have raise the OSS message about this but didnt find any solution yet. SAP asked to perform temse and BTC processing check but issue still persist. Please help me as it happen quite frequent now.
Regards
Aiman Alimi