ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » Subpool server has terminated while application connected

Post new topic  Reply to topic Goto page Previous  1, 2
 Subpool server has terminated while application connected « View previous topic :: View next topic » 
Author Message
jhidalgo
PostPosted: Tue Jan 20, 2009 12:04 pm    Post subject: Reply with quote

Disciple

Joined: 26 Mar 2008
Posts: 161
Can anybody help me with the interpretation of this segment:

Code:

MQM Trace History
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=OK
-------} zcpReceiveOnPipe rc=OK
------} ziiSendReceiveAgent rc=arcE_NO_MSG_AVAILABLE
------{ zcpDeleteMessage
-------{ xcsFreeMemBlock
-------} xcsFreeMemBlock rc=OK
------} zcpDeleteMessage rc=OK
-----} ziiMQGET rc=OK
----} zstMQGET rc=arcE_NO_MSG_AVAILABLE
----{ MQCheckUOWMix
----} MQCheckUOWMix rc=arcE_NO_MSG_AVAILABLE
---} lpiSPIGet rc=arcE_NO_MSG_AVAILABLE
---{ rriConfirm
----{ ccxSend
-----{ cciTcpSend
------{ send
------} send rc=Unknown(1C)
-----} cciTcpSend rc=OK
----} ccxSend rc=OK
----{ ccxFreeMem
----} ccxFreeMem rc=OK
----{ ccxReceive
-----{ cciTcpReceive
------{ ccxAllocMem
------} ccxAllocMem rc=OK
------{ recv
------} recv rc=Unknown(FFFF)
------{ xcsWaitFd
-------{ select
-------} select rc=Unknown(1)
------} xcsWaitFd rc=Unknown(1)
------{ recv
------} recv rc=Unknown(1C)
-----} cciTcpReceive rc=OK
----} ccxReceive rc=OK
----{ ccxFreeMem
----} ccxFreeMem rc=OK
----{ xcsQueryValueForSubpool
----} xcsQueryValueForSubpool rc=OK
---} rriConfirm rc=OK
---{ ccxAllocMem
---} ccxAllocMem rc=OK
---{ lpiSPIGet
----{ zstMQGET
-----{ zstVerifyPCD
-----} zstVerifyPCD rc=OK
-----{ ziiMQGET
------{ ziiCreateIPCCMessage
-------{ zcpCreateMessage
-------} zcpCreateMessage rc=OK
------} ziiCreateIPCCMessage rc=OK
------{ ziiSendReceiveAgent
-------{ zcpSendOnPipe
--------{ xcsResetEventSem
--------} xcsResetEventSem rc=OK
--------{ xcsPostEventSem
--------} xcsPostEventSem rc=OK
-------} zcpSendOnPipe rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=xecP_E_INVALID_PID
-------} ziiHealthCheck rc=xecP_E_INVALID_PID
------} ziiSendReceiveAgent rc=lrcE_CONNECTION_BROKEN
-----} ziiMQGET rc=xecP_E_INVALID_PID
-----{ ziiBreakConnection
------{ zcpDetachPipe
-------{ xcsDisconnectSharedMemSet
--------{ xcsRequestThreadMutexSem
--------} xcsRequestThreadMutexSem rc=OK
--------{ xcsFreeMem
--------} xcsFreeMem rc=OK
--------{ xcsFreeMem
--------} xcsFreeMem rc=OK
--------{ xstDisconnectExtent
---------{ xcsRequestThreadMutexSem
---------} xcsRequestThreadMutexSem rc=OK
---------{ xstStorageRequest
----------{ xstServerRequest
-----------{ xcsFFST
 


I think it's all about that part
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Jan 20, 2009 2:41 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20767
Location: LI,NY
jhidalgo wrote:
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=OK
-------} ziiHealthCheck rc=OK
-------{ zcpReceiveOnPipe
--------{ xcsWaitEventSem
--------} xcsWaitEventSem rc=xecL_W_TIMEOUT
-------} zcpReceiveOnPipe rc=zrcC_E_TIMEOUT
-------{ ziiHealthCheck
--------{ xcsCheckHandle
--------} xcsCheckHandle rc=xecP_E_INVALID_PID
-------} ziiHealthCheck rc=xecP_E_INVALID_PID
------} ziiSendReceiveAgent rc=lrcE_CONNECTION_BROKEN
-----} ziiMQGET rc=xecP_E_INVALID_PID
-----{ ziiBreakConnection

Looks quite clear to me, may be something on the tcp/ip layer? Or did someone terminate a process (kill -9 )?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
jhidalgo
PostPosted: Tue Jan 20, 2009 3:15 pm    Post subject: Reply with quote

Disciple

Joined: 26 Mar 2008
Posts: 161
It's a 2003 box, what happens is that this error happened many many times so nobody could be killing the processes that fast. I didnt find anything related to TCP in the Event Viewer.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Jan 20, 2009 3:24 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20767
Location: LI,NY
jhidalgo wrote:
It's a 2003 box, what happens is that this error happened many many times so nobody could be killing the processes that fast. I didnt find anything related to TCP in the Event Viewer.

Windows ... what a surprise. Maybe a resource problem and the task manager playing crazy trying to free up resources....
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
jhidalgo
PostPosted: Tue Jan 20, 2009 3:50 pm    Post subject: Reply with quote

Disciple

Joined: 26 Mar 2008
Posts: 161
the problem is that Windows is not recording anything on Event Viewer, and MQ is not logging any problem about related to "out of memory".
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Jan 20, 2009 4:26 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447
Have you considered increasing the JVM heap for the WAS Application Server instance that is making a bindings connection to MQ, and therefore running portions of it's code inside the Qmgr memory space, and causing the Qmgr to throw an FDC when that portion of WAS tries to get more memory?
Back to top
View user's profile Send private message
jhidalgo
PostPosted: Tue Jan 20, 2009 7:01 pm    Post subject: Reply with quote

Disciple

Joined: 26 Mar 2008
Posts: 161
how can I tell if that is my problem ?, I can see some amqzlaa0.exe taking > 400mb sometimes, I don't know if that may be related to what you said.

Makes sense to me in regards to

Code:

| Probe Description :- AMQ6119: An internal WebSphere MQ error has occurred   |
|   (Failed to complete request to Shared Storage Server)                     |
| FDCSequenceNumber :- 1                                                      |
| Comment1          :- Failed to complete request to Shared Storage Server 


But I don't know how to dig those issues...
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jan 21, 2009 1:41 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
fjb_saper wrote:
Maybe a resource problem and the task manager playing crazy trying to free up resources....



_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jan 21, 2009 1:43 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
jhidalgo wrote:
But I don't know how to dig those issues...


I think, as was said further up this thread, that WMQ is more a victim than a perpitrator in all of this.

I'd attack the problem from the other angle; why is the Shared Storage Server unresponsive? That's a question for MSoft and their knowledge base.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
jhidalgo
PostPosted: Thu Jan 22, 2009 1:01 pm    Post subject: Reply with quote

Disciple

Joined: 26 Mar 2008
Posts: 161
the final recommendation they gave me is:

Quote:
Start the qmgr with amqmdain so all the processes will run as threads and the AMQXSSVN (shared memory server) will always be available, this means one huge process with lots of threads.


I just made a test and amqmdain seems to run the same separe processes as strmqm.....

but if this were true! will lead me to another problem, since I am using 32bits the 2Gb limitation will kill the whole qmgr at once !

I'm not very happy with this 'solution'
Back to top
View user's profile Send private message
WBI_User21
PostPosted: Tue Feb 03, 2009 3:00 pm    Post subject: Reply with quote

Voyager

Joined: 12 Jun 2007
Posts: 98
Some time back ,i got same error with Websphere MQ in Windows
I got similar response from IBM. I shared same with forum..

http://www.mqseries.net/phpBB2/viewtopic.php?t=46010
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Feb 04, 2009 3:59 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20767
Location: LI,NY
Maybe it's time to move off the win platform for this qmgr...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
WBI_User21
PostPosted: Wed Feb 04, 2009 7:28 am    Post subject: Reply with quote

Voyager

Joined: 12 Jun 2007
Posts: 98
exactly fjb_saper..

Next immediate step take by application team was moving QMGR to Solaris from Window..
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page Previous  1, 2 Page 2 of 2

MQSeries.net Forum Index » General IBM MQ Support » Subpool server has terminated while application connected
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
  
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.