ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » Workflow Engines - IBM MQ Workflow & Business Process Choreographer » DB2 DEADLOCKS OCCURS WHEN USING PROCESS MONITOR

Post new topic  Reply to topic
 DB2 DEADLOCKS OCCURS WHEN USING PROCESS MONITOR « View previous topic :: View next topic » 
Author Message
SABCAS
PostPosted: Thu May 22, 2008 8:41 am    Post subject: DB2 DEADLOCKS OCCURS WHEN USING PROCESS MONITOR Reply with quote

Acolyte

Joined: 09 Mar 2005
Posts: 60
Location: Switzerland
Hallo
I use the MQ Workflow 3.6 + SP4 on AIX, and I have a Problem with one Process when I try to use the processMonitor
WEBCLI1_exception.log
Index Occur Time of last Occurence Exception SourceId ProbeId
ences
-----------------------------------------------------------------------
3 10905 5/20/08 12:58:05:703 CEST java.io.FileNotFoundException com.ibm.ws.webcontainer.servlet.SimpleFileServlet.doGet 312
13 2109 5/20/08 12:56:28:246 CEST java.net.SocketException com.ibm.ws.webcontainer.srt.BufferedWriter.writeOut 416
20 93 5/20/08 12:56:28:279 CEST java.net.SocketException com.ibm.ws.webcontainer.srt.BufferedWriter.flushChars 409

fmcsys.log
05/20/08 13:03:43 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:03:54 FMC31100W The message InvokeProgramResponse could not be processed because of a database deadlock or timeout. The message will be retried.
05/20/08 13:03:54 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:04:05 FMC31100W The message InvokeProgramResponse could not be processed because of a database deadlock or timeout. The message will be retried.
05/20/08 13:04:06 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:04:51 FMC31100W The message InvokeProgramResponse could not be processed because of a database deadlock or timeout. The message will be retried.
05/20/08 13:04:51 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:05:42 FMC31100W The message InvokeProgramResponse could not be processed because of a database deadlock or timeout. The message will be retried.
05/20/08 13:05:42 FMC31130W Detected packages older then the database statistics. *** Rebind database packages. ***
05/20/08 13:05:51 FMC31059W The retry limit for message "QryProcInstStatus" from "RTC" was exceeded. The message cannot be processed and it has been ignored.
05/20/08 13:08:04 FMC12240E Execution server instance(s) stopped with an error.

SystemOut.log
[5/20/08 13:19:15:912 CEST] 7da746a2 WebGroup I SRVE0181I: [IBM WebSphere MQ Workflow Web Client] [/MQWFClient-WEBCLI2] [Servlet.LOG]: doGet (showMonitor): com.ibm.workflow.servlet.client.ClientException: FMC00014E Timeout occurred: com.ibm.workflow.client.util.FmcApiException:
FMC38009E MQSeries Workflow API Error :
API Return Code : 14
Error Origin : FmcFMLC.java
Error Message : FMC00014E Timeout occurred
Nested Exception : None
at com.ibm.workflow.client.util.FmcFMLC.GetMsg(FmcFMLC.java(Compiled Code))
at com.ibm.workflow.client.util.FmcFMLC.Converse(FmcFMLC.java(Compiled Code))
at com.ibm.workflow.client.api.FmcSession.CallSync(FmcSession.java(Compiled Code))
at com.ibm.workflow.client.api.FmcProcessInstance.obtainProcessMonitor(FmcProcessInstance.java(Compiled Code))
at com.ibm.workflow.servlet.client.ProcessMonitor.drawMonitor(ProcessMonitor.java(Inlined Compiled Code))
at com.ibm.workflow.servlet.client.MonitorHandler.showMonitor(MonitorHandler.java(Compiled Code))
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java(Compiled Code))
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java(Compiled Code))
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled Code))
at java.lang.reflect.Method.invoke(Method.java(Compiled Code))
at com.ibm.workflow.servlet.client.Util.invoke(Util.java(Compiled Code))
at com.ibm.workflow.servlet.client.GenericCommandHandler.execute(GenericCommandHandler.java(Compiled Code))
at com.ibm.workflow.servlet.client.BuiltinHandler.execute(BuiltinHandler.java(Compiled Code))
at com.ibm.workflow.servlet.client.ApplicationContextImpl.dispatch(ApplicationContextImpl.java(Inlined Compiled Code))
at com.ibm.workflow.servlet.client.Main.doGet(Main.java(Compiled Code))
at javax.servlet.http.HttpServlet.service(HttpServlet.java(Compiled Code))
at javax.servlet.http.HttpServlet.service(HttpServlet.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.StrictServletInstance.doService(StrictServletInstance.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet._service(StrictLifecycleServlet.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.IdleServletState.service(StrictLifecycleServlet.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet.service(StrictLifecycleServlet.java(Inlined Compiled Code))
at com.ibm.ws.webcontainer.servlet.ServletInstance.service(ServletInstance.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.ValidServletReferenceState.dispatch(ValidServletReferenceState.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.ServletInstanceReference.dispatch(ServletInstanceReference.java(Inlined Compiled Code))
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java(Compiled Code))
at com.swisscom.sifmsg.webclient.flow4web.NSIFilter.doFilter(NSIFilter.java(Compiled Code))
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java(Compiled Code))
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java(Compiled Code))
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.handleWebAppDispatch(WebAppRequestDispatcher.java(Compiled Code))
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.dispatch(WebAppRequestDispatcher.java(Compiled Code))
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.forward(WebAppRequestDispatcher.java(Compiled Code))
at com.ibm.ws.webcontainer.srt.WebAppInvoker.doForward(WebAppInvoker.java(Compiled Code))
at com.ibm.ws.webcontainer.srt.WebAppInvoker.handleInvocationHook(WebAppInvoker.java(Compiled Code))
at com.ibm.ws.webcontainer.cache.invocation.CachedInvocation.handleInvocation(CachedInvocation.java(Compiled Code))
at com.ibm.ws.webcontainer.cache.invocation.CacheableInvocationContext.invoke(CacheableInvocationContext.java(Compiled Code))
at com.ibm.ws.webcontainer.srp.ServletRequestProcessor.dispatchByURI(ServletRequestProcessor.java(Compiled Code))
at com.ibm.ws.webcontainer.oselistener.OSEListenerDispatcher.service(OSEListener.java(Compiled Code))
at com.ibm.ws.webcontainer.http.HttpConnection.handleRequest(HttpConnection.java(Compiled Code))
at com.ibm.ws.http.HttpConnection.readAndHandleRequest(HttpConnection.java(Compiled Code))
at com.ibm.ws.http.HttpConnection.run(HttpConnection.java(Compiled Code))
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java(Compiled Code))

I saw on IBM WebSite This Problem was with SP3 fixed.
What I check is that the Process has State= RUNNING and an Activity is started, after an Application send the following Message to set the Activity into Error:
<?xml version="1.0" encoding="ISO-8859-1"?>
<WfMessage>
<WfMessageHeader>
<ResponseRequired>IfError</ResponseRequired>
<UserContext>ADMIN</UserContext>
</WfMessageHeader>
<ActivityImplInvokeResponse>
<ActImplCorrelID>RUEAAAABN/DE6QAAAAIAAAAAAAAAKAAAAAEJ6sAAAAAAAAAAAAAAAAAoQQAAAAE8mIBJAAAAAgAAAABF</ActImplCorrelID>
<Exception>
<Rc>1113</Rc>
<Parameters>
<Parameter>262337 ; F;IPSS;invalid content of Business Object;soap:Client null System.Web.Services.Protocols.SoapException: Server was unable to read request. ---&gt; System.InvalidOperationException: There is an error in XML document (2, 178). ---&gt; System.OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown. --- End of inner exception stack trace --- at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events) at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, XmlDeserializationEvents events) at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader) at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters() --- End of inner exception stack trace --- at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters() at System.Web.Services.Protocols.WebServiceHandler.Invoke() at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()</Parameter>
<Parameter/>
<null/>
</Parameters>
<MessageText/>
<Origin>com.swisscom.sif.oe.dto.IpsActivityImpl</Origin>
</Exception>
</ActivityImplInvokeResponse>
</WfMessage>
After the MQ Workflow received this Message, when I try with ProcessMonitor I receive the Timeout and all ExecutionServer would be restarted.
Somebody could tell me, how could I unlock the Process ?
Thanks a Lot
Ciao
Sabato
Back to top
View user's profile Send private message
hos
PostPosted: Mon May 26, 2008 12:24 am    Post subject: Reply with quote

Chevalier

Joined: 03 Feb 2002
Posts: 470
Hi,

I think the error and warning messages are perfectly fine - just do what they suggest: start the administration utility fmcautil and rebind you DB packages.
Your API call times out because it could not be processed due to this DB problem. Not sure about the problem that eventually brought your server down , but it is very likely that this is also related to the outdated packages.
Rebind and then retry your scenario. By the way: MQWF needs DB maintenance and optimization as any other product that has a DB (e.g. backup, runstat, etc.).
Back to top
View user's profile Send private message
SABCAS
PostPosted: Mon May 26, 2008 1:29 am    Post subject: Reply with quote

Acolyte

Joined: 09 Mar 2005
Posts: 60
Location: Switzerland
Hi,
yes we used fmczutil to make the rbind of DB2, and it was succesfully terminated, but I still have a timeout when I try on the Process to see the ProcessMonitor.

thanks a Lot
Ciao
Sabato
Back to top
View user's profile Send private message
hos
PostPosted: Mon May 26, 2008 6:09 am    Post subject: Reply with quote

Chevalier

Joined: 03 Feb 2002
Posts: 470
Well,
if the warning messages with respect to the outdated packages are gone now and you still see a termination of the execution servers once you call this API via the web client, I recommend to open a PMR. No API call must bring down the server. You can also check the error log fmcerr.log whether it contains additional information about the error that brings the servers down.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » Workflow Engines - IBM MQ Workflow & Business Process Choreographer » DB2 DEADLOCKS OCCURS WHEN USING PROCESS MONITOR
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
  
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.