| Author |
Message
|
| yupoet |
 Posted: Thu May 24, 2012 8:08 am Post subject: AS400 - Long recovery from Journal after copy disk to disk |
 |
|
Apprentice
Joined: 26 Nov 2008
Posts: 36
|
We did a hardisk to hardisk copy (just like clone) to migrate MQ system from one AS400 system to another.
It took a long time (5 hours) to start one queue manager, out of totally 3. The other 2 is ok, took about few minutes.
We want to know why it took such a long time to start for the certain queue manager.
_________________
With warm regards and thanks! |
|
| Back to top |
|
 |
| yupoet |
| Posted: Thu May 24, 2012 8:09 am Post subject: wrkobj AMQZXMAX |
|
|
Apprentice
Joined: 26 Nov 2008
Posts: 36
|
We did a wrkjob AMQZXMAX and found the spool files log -->
CPF7073 Escape 40 05/23/12 11:17:21.227160 QJORTVJE
To module . . . . . . . . . : AMQAJSRX
To procedure . . . . . . . : AMQAJSRX
Statement . . . . . . . . . : 214
Message . . . . : No entry retrieved fr
Cause . . . . . : There was no journal
(RCVRNG) that satisfied the specified s
Change the value for the RCVRNG, the FR
values. Then try the request again.
AMQ6125 Diagnostic 40 05/23/12 11:17:21.436512 LIBMQMCS
From module . . . . . . . . : AMQXEIMX_
From procedure . . . . . . : xcsSendMe
Statement . . . . . . . . . : 38
To module . . . . . . . . . : QP0ZPCPN
To procedure . . . . . . . : InvokeTar
Statement . . . . . . . . . : 210
Message . . . . : An internal WebSphere
Cause . . . . . : An internal error has
X'20800893'. This message is issued in
Recovery . . . : Use the standard fa
record the problem identifier, and to s
Contact your IBM support center. Do no
problem has been resolved. Technical
5722SS1 V5R4M0 060210 Job Log
Job name . . . . . . . . . . : AMQZXMAX User . . . . . . : QMQM
Job description . . . . . . : AMQZXMA0 Library . . . . . : QMQM
MSGID TYPE SEV DATE TIME FROM PGM
AMQ6184 Diagnostic 10 05/23/12 11:17:21.437840 LIBMQMCS
From module . . . . . . . . : AMQXEIMX_
From procedure . . . . . . : xcsSendMe
Statement . . . . . . . . . : 38
To module . . . . . . . . . : QP0ZPCPN
To procedure . . . . . . . : InvokeTar
Statement . . . . . . . . . : 210
Message . . . . : An internal WebSphere
Cause . . . . . : An internal MQ error
ASMQMPRICN and the MQ error recording r
process is process 293. Recovery . .
supplied with your system to record the
CPF9861 Information 00 05/23/12 12:16:43.243952 QLIOUTFL
To module . . . . . . . . . : AMQXPSAX_
To procedure . . . . . . . : xcsInitQM
Statement . . . . . . . . . : 45
Message . . . . : Output file P00000029
CPF9862 Information 00 05/23/12 12:16:43.261224 QLIOUTFL
To module . . . . . . . . . : AMQXPSAX_
To procedure . . . . . . . : xcsInitQM
Statement . . . . . . . . . : 45
Message . . . . : Member P000000293 add
library QMQM.
CPF9861 Information 00 05/23/12 12:16:43.282712 QLIOUTFL
To module . . . . . . . . . : AMQXPSAX_
(sorry not easy to copy the full log, the right side has been truncated)
_________________
With warm regards and thanks! |
|
| Back to top |
|
|
| yupoet |
| Posted: Thu May 24, 2012 8:11 am Post subject: FDC AL057001 with arcE_LOG_RECD_NOT_FOUND |
|
|
Apprentice
Joined: 26 Nov 2008
Posts: 36
|
And there is two FDC related to journal
+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Wednesday May 23 11:17:21 Beijing Standard Time 2012 |
| Host Name :- CNI400B1.CMC-XINNUO.COM (OS400 V5R4M0) |
| PIDS :- 5724H7206 |
| LVLS :- 6.0.2.8 |
| Product Long Name :- WebSphere MQ for iSeries |
| Vendor :- IBM |
| Probe Id :- AL057001 |
| Application Name :- MQM |
| Component :- alsDoReadLog |
| Component :- alsDoReadLog |
| SCCS Info :- lib/lqm/unix/as400/amqalrcx.c, 1.47.1.13 |
| Line Number :- 3262 |
| Build Date :- Oct 19 2009 |
| CMVC level :- p600-208-090930 |
| UserID :- 00000108 (QMQM) |
| Job Name :- 004068/QMQM/AMQZXMAX |
| Job Description :- QMQM/AMQZXMA0 |
| Submitted By :- 004065/QMQM/AMQZXMA0 |
| Activation Group :- 17 (QMQM) (QMQM/AMQZXMAX) |
| Max File Handles :- 2048 |
| Process :- 00000293 |
| QueueManager :- ASMQMPRICN |
| ConnId 1 IPCC :- 5 |
| ConnId 2 QM :- 5 |
| ConnId 2 QM :- 5 |
| ConnId 3 QM-P :- 4 |
| Major Errorcode :- krcE_UNEXPECTED_ERROR |
| Minor Errorcode :- OK |
| Probe Type :- INCORROUT |
| Probe Severity :- 2 |
| Probe Description :- AMQ6125: An internal WebSphere MQ error has occurred. |
| FDCSequenceNumber :- 0 |
| Arith1 :- 0 |
| Arith2 :- 0 |
| Comment1 :- CPF6946 |
| |
| Comment2 :- AMQAJRN QMASMQMPRI |
| |
| |
| |
+-----------------------------------------------------------------------------+
MQM Function Stack
kpiStartup
apiStartup
almPerformReDoPass
alsLocateReplayJSN
alsRetrieveLog
alsReadLog
alsDoReadLog
xcsFFST
MQM Trace History
--> aloReadLog
--> aloReadLog
<-- aloReadLog rc=OK
--> aloReadLog
<-- aloReadLog rc=OK
<-- alsFindReceiver rc=OK
......
<-- xcsReleaseMutexSem rc=OK
--> alsRetrieveLog
--> alsReadLog
--> alsDoReadLog
--> xcsGetMem
<-- xcsGetMem rc=OK
--> aloReadLog
<-- aloReadLog rc=arcE_LOG_RECD_NOT_FOUND
--> xcsFreeMem
<-- xcsFreeMem rc=OK
<-- alsDoReadLog rc=arcE_LOG_RECD_NOT_FOUND
--> alsDoReadLog
......
<-- alsDoReadLog rc=arcE_LOG_RECD_NOT_FOUND
<-- alsReadLog rc=arcE_LOG_RECD_NOT_FOUND
--> alsReadLog
--> alsDoReadLog
--> xcsGetMem
<-- xcsGetMem rc=OK
ExceptID
SPP:0000 :1aefAMQZXMAX QMQM 004068 :17f0:0:11
SPP:0x000017f0: C3D7C6F6 F9F4F6 CPF6946
_________________
With warm regards and thanks!
Last edited by yupoet on Thu May 24, 2012 8:57 pm; edited 1 time in total |
|
| Back to top |
|
|
| yupoet |
| Posted: Thu May 24, 2012 8:14 am Post subject: |
|
|
Apprentice
Joined: 26 Nov 2008
Posts: 36
|
The background is --> the users have never performed a stop/start before until this time migration. And he/she have never performed any journal management activities.
I have checked the journal status
QMASMQMPRI
Work with Journal Attributes
Journal . . . . . . : AMQAJRN Library . . . . . . : QMASMQMPRI
Attached receiver . : AMQA101126 Library . . . . . . : QMASMQMPRI
Text . . . . . . . . : MQM local journal
ASP . . . . . . . . : 1 Journaled objects:
Message queue . . . : AMQAJRNMSG Current . . . . . : 293
Library . . . . . : QMASMQMPRI Maximum . . . . . : 250000
Manage receivers . . : *SYSTEM Recovery count . . . : *SYSDFT
Delete receivers . . : *NO Receiver size options: *MAXOPT2
Journal cache . . . : *NO Fixed length data . : *JOB
Manage delay . . . . : 10 *USR
Delete delay . . . . : 10 *PGM
Journal type . . . . : *LOCAL
Journal state . . . : *ACTIVE
Minimize entry data : *NONE
Journal . . . . . . : AMQAJRN Library . . . . . . : QMASMQMPRI
Last system end status . . . . . . . . . . . . . : Normal
Journal damage status . . . . . . . . . . . . . . : None
All objects synchronized . . . . . . . . . . . . : Yes
Attached Damage
Receiver Library Status
AMQA101126 QMASMQMPRI None
The oldest one is
Journey Receiver AMQA000835
The newest one is
Journey Receiver AMQA101126
_________________
With warm regards and thanks! |
|
| Back to top |
|
|
| yupoet |
| Posted: Thu May 24, 2012 8:20 am Post subject: |
|
|
Apprentice
Joined: 26 Nov 2008
Posts: 36
|
My rough conclusion is that , the MQ found something missing (maybe the Queue Manager was not shut down correctly), and then trying to recover from checkpoints, hence lots of journal have to be read, which consumes long time.
While I am still puzzled. A few queries for the you MQ Heroes:
1. Any idea why something is missing and why a long journal read is needed? Compared to other 2 non-busy queue managers.
2. To avoid this long outage next time, can I issue this first to force a checkpoint?
RCDMQMIMG OBJ(*ALL) OBJTYPE(*ALL) MQMNAME(ASMQMPRI) DSPJRNDATA(*YES)
3. Is a cold start required for fixing this issue permanently?
http://www-304.ibm.com/support/docview.wss?uid=swg21140850
_________________
With warm regards and thanks! |
|
| Back to top |
|
|
| exerk |
| Posted: Thu May 24, 2012 8:45 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
As a courtesy, would you kindly remove all but the header information from the FDC you posted as the rest of the information is not of use to anyone but IBM support. Thank you.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| yupoet |
| Posted: Thu May 24, 2012 8:59 pm Post subject: |
|
|
Apprentice
Joined: 26 Nov 2008
Posts: 36
|
Thank you for the comments.
I have removed some logs which might not be useful. Still I left a bit on the page, in case any of them are informational.
_________________
With warm regards and thanks! |
|
| Back to top |
|
|
|
|
