|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
|
|
DB2 DEADLOCKS OCCURS WHEN USING PROCESS MONITOR |
« View previous topic :: View next topic » |
Author |
Message
|
SABCAS |
Posted: Thu May 22, 2008 8:41 am Post subject: DB2 DEADLOCKS OCCURS WHEN USING PROCESS MONITOR |
|
|
Acolyte
Joined: 09 Mar 2005 Posts: 60 Location: Switzerland
|
Hallo
I use the MQ Workflow 3.6 + SP4 on AIX, and I have a Problem with one Process when I try to use the processMonitor
WEBCLI1_exception.log
Index Occur Time of last Occurence Exception SourceId ProbeId
ences
-----------------------------------------------------------------------
3 10905 5/20/08 12:58:05:703 CEST java.io.FileNotFoundException com.ibm.ws.webcontainer.servlet.SimpleFileServlet.doGet 312
13 2109 5/20/08 12:56:28:246 CEST java.net.SocketException com.ibm.ws.webcontainer.srt.BufferedWriter.writeOut 416
20 93 5/20/08 12:56:28:279 CEST java.net.SocketException com.ibm.ws.webcontainer.srt.BufferedWriter.flushChars 409
fmcsys.log
05/20/08 13:03:43 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:03:54 FMC31100W The message InvokeProgramResponse could not be processed because of a database deadlock or timeout. The message will be retried.
05/20/08 13:03:54 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:04:05 FMC31100W The message InvokeProgramResponse could not be processed because of a database deadlock or timeout. The message will be retried.
05/20/08 13:04:06 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:04:51 FMC31100W The message InvokeProgramResponse could not be processed because of a database deadlock or timeout. The message will be retried.
05/20/08 13:04:51 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:05:42 FMC31100W The message InvokeProgramResponse could not be processed because of a database deadlock or timeout. The message will be retried.
05/20/08 13:05:42 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:05:51 FMC31059W The retry limit for message "QryProcInstStatus" from "RTC" was exceeded. The message cannot be processed and it has been ignored.
05/20/08 13:08:04 FMC12240E Execution server instance(s) stopped with an error.
SystemOut.log
[5/20/08 13:19:15:912 CEST] 7da746a2 WebGroup I SRVE0181I: [IBM WebSphere MQ Workflow Web Client] [/MQWFClient-WEBCLI2] [Servlet.LOG]: doGet (showMonitor): com.ibm.workflow.servlet.client.ClientException: FMC00014E Timeout occurred: com.ibm.workflow.client.util.FmcApiException:
FMC38009E MQSeries Workflow API Error :
API Return Code : 14
Error Origin : FmcFMLC.java
Error Message : FMC00014E Timeout occurred
Nested Exception : None
at com.ibm.workflow.client.util.FmcFMLC.GetMsg(FmcFMLC.java(Compiled Code))
at com.ibm.workflow.client.util.FmcFMLC.Converse(FmcFMLC.java(Compiled Code))
at com.ibm.workflow.client.api.FmcSession.CallSync(FmcSession.java(Compiled Code))
at com.ibm.workflow.client.api.FmcProcessInstance.obtainProcessMonitor(FmcProcessInstance.java(Compiled Code))
at com.ibm.workflow.servlet.client.ProcessMonitor.drawMonitor(ProcessMonitor.java(Inlined Compiled Code))
at com.ibm.workflow.servlet.client.MonitorHandler.showMonitor(MonitorHandler.java(Compiled Code))
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java(Compiled Code))
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java(Compiled Code))
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled Code))
at java.lang.reflect.Method.invoke(Method.java(Compiled Code))
at com.ibm.workflow.servlet.client.Util.invoke(Util.java(Compiled Code))
at com.ibm.workflow.servlet.client.GenericCommandHandler.execute(GenericCommandHandler.java(Compiled Code))
at com.ibm.workflow.servlet.client.BuiltinHandler.execute(BuiltinHandler.java(Compiled Code))
at com.ibm.workflow.servlet.client.ApplicationContextImpl.dispatch(ApplicationContextImpl.java(Inlined Compiled Code))
at com.ibm.workflow.servlet.client.Main.doGet(Main.java(Compiled Code))
at javax.servlet.http.HttpServlet.service(HttpServlet.java(Compiled Code))
at javax.servlet.http.HttpServlet.service(HttpServlet.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.StrictServletInstance.doService(StrictServletInstance.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet._service(StrictLifecycleServlet.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.IdleServletState.service(StrictLifecycleServlet.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet.service(StrictLifecycleServlet.java(Inlined Compiled Code))
at com.ibm.ws.webcontainer.servlet.ServletInstance.service(ServletInstance.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.ValidServletReferenceState.dispatch(ValidServletReferenceState.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.ServletInstanceReference.dispatch(ServletInstanceReference.java(Inlined Compiled Code))
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java(Compiled Code))
at com.swisscom.sifmsg.webclient.flow4web.NSIFilter.doFilter(NSIFilter.java(Compiled Code))
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java(Compiled Code))
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java(Compiled Code))
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.handleWebAppDispatch(WebAppRequestDispatcher.java(Compiled Code))
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.dispatch(WebAppRequestDispatcher.java(Compiled Code))
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.forward(WebAppRequestDispatcher.java(Compiled Code))
at com.ibm.ws.webcontainer.srt.WebAppInvoker.doForward(WebAppInvoker.java(Compiled Code))
at com.ibm.ws.webcontainer.srt.WebAppInvoker.handleInvocationHook(WebAppInvoker.java(Compiled Code))
at com.ibm.ws.webcontainer.cache.invocation.CachedInvocation.handleInvocation(CachedInvocation.java(Compiled Code))
at com.ibm.ws.webcontainer.cache.invocation.CacheableInvocationContext.invoke(CacheableInvocationContext.java(Compiled Code))
at com.ibm.ws.webcontainer.srp.ServletRequestProcessor.dispatchByURI(ServletRequestProcessor.java(Compiled Code))
at com.ibm.ws.webcontainer.oselistener.OSEListenerDispatcher.service(OSEListener.java(Compiled Code))
at com.ibm.ws.webcontainer.http.HttpConnection.handleRequest(HttpConnection.java(Compiled Code))
at com.ibm.ws.http.HttpConnection.readAndHandleRequest(HttpConnection.java(Compiled Code))
at com.ibm.ws.http.HttpConnection.run(HttpConnection.java(Compiled Code))
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java(Compiled Code))
I saw on IBM WebSite This Problem was with SP3 fixed.
What I check is that the Process has State= RUNNING and an Activity is started, after an Application send the following Message to set the Activity into Error:
<?xml version="1.0" encoding="ISO-8859-1"?>
<WfMessage>
<WfMessageHeader>
<ResponseRequired>IfError</ResponseRequired>
<UserContext>ADMIN</UserContext>
</WfMessageHeader>
<ActivityImplInvokeResponse>
<ActImplCorrelID>RUEAAAABN/DE6QAAAAIAAAAAAAAAKAAAAAEJ6sAAAAAAAAAAAAAAAAAoQQAAAAE8mIBJAAAAAgAAAABF</ActImplCorrelID>
<Exception>
<Rc>1113</Rc>
<Parameters>
<Parameter>262337 ; F;IPSS;invalid content of Business Object;soap:Client null System.Web.Services.Protocols.SoapException: Server was unable to read request. ---> System.InvalidOperationException: There is an error in XML document (2, 178). ---> System.OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown. --- End of inner exception stack trace --- at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events) at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, XmlDeserializationEvents events) at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader) at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters() --- End of inner exception stack trace --- at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters() at System.Web.Services.Protocols.WebServiceHandler.Invoke() at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()</Parameter>
<Parameter/>
<null/>
</Parameters>
<MessageText/>
<Origin>com.swisscom.sif.oe.dto.IpsActivityImpl</Origin>
</Exception>
</ActivityImplInvokeResponse>
</WfMessage>
After the MQ Workflow received this Message, when I try with ProcessMonitor I receive the Timeout and all ExecutionServer would be restarted.
Somebody could tell me, how could I unlock the Process ?
Thanks a Lot
Ciao
Sabato |
|
Back to top |
|
|
hos |
Posted: Mon May 26, 2008 12:24 am Post subject: |
|
|
Chevalier
Joined: 03 Feb 2002 Posts: 470
|
Hi,
I think the error and warning messages are perfectly fine - just do what they suggest: start the administration utility fmcautil and rebind you DB packages.
Your API call times out because it could not be processed due to this DB problem. Not sure about the problem that eventually brought your server down , but it is very likely that this is also related to the outdated packages.
Rebind and then retry your scenario. By the way: MQWF needs DB maintenance and optimization as any other product that has a DB (e.g. backup, runstat, etc.). |
|
Back to top |
|
|
SABCAS |
Posted: Mon May 26, 2008 1:29 am Post subject: |
|
|
Acolyte
Joined: 09 Mar 2005 Posts: 60 Location: Switzerland
|
Hi,
yes we used fmczutil to make the rbind of DB2, and it was succesfully terminated, but I still have a timeout when I try on the Process to see the ProcessMonitor.
thanks a Lot
Ciao
Sabato |
|
Back to top |
|
|
hos |
Posted: Mon May 26, 2008 6:09 am Post subject: |
|
|
Chevalier
Joined: 03 Feb 2002 Posts: 470
|
Well,
if the warning messages with respect to the outdated packages are gone now and you still see a termination of the execution servers once you call this API via the web client, I recommend to open a PMR. No API call must bring down the server. You can also check the error log fmcerr.log whether it contains additional information about the error that brings the servers down. |
|
Back to top |
|
|
|
|
|
|
Page 1 of 1 |
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|
|
|