| Author |
Message
|
| jbanoop |
 Posted: Tue Nov 10, 2009 3:26 pm Post subject: Irratic slowness with message flow having mapping node |
 |
|
Chevalier
Joined: 17 Sep 2005
Posts: 401
Location: SC
|
All,
Env: WMB 6.1.0.3, AIX, MQ 6.0
Toolkit version : 6.1.0 (Build Id: 6.1.0-20071108_1112)
We recently had a peculiar issue pertaining to a particular message flow.
The flow is pretty straightforward and does a mapping from an IDOC format to a copybook format using the mapping node.
There are no subroutines being called from within the mapping node.
The flow was processing normally until suddenly the performance decreased drastically. The flow was processing at a rate of 1 msg/sec atleast and it reduced to 1Msg/5 mins. The dataflow engine was hogging the CPU (read 90-100% cpu usage) as well.
stats run on the flow revealed that the bulk of the processing time was spent in the mapping node.
unsuccessful options tried to rectify the issue included:
* reload of the execution group
* restart the broker
* Redeployment of same code in a new bar file.
Finally the issue was resolved when we deployed an older bar file having the exact same code. So atleast on the face of it, it seems to be some issue with how the specific bar file is getting created.
Any directions on how to analyze / identify / pinpoint the issue would be very helpful.
Also it would be great to know if anyone else has run into similar issues and if so how this was resolved.
Regards, |
|
| Back to top |
|
 |
| kirani |
| Posted: Tue Nov 10, 2009 4:06 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
I'd look at following thigs:
1. Find out if the issue was caused by input data?
2. Look at the modified timestamp, version and size of the compiled message flow file in your "good" and "bad" bar file and compare then.
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Wed Nov 11, 2009 1:00 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
Also, If you redeploy the "bad" bar file does the problem come back? I suspect not as your post seems to suggest that this code was running for a while without issue but it would be interesting to try anyway.
How many flow instances do you have? Could it be an instance was loopingng in the mapping node and hogging resource while the other carried on with the limited remaining resources, at the slower rate?
Just out of interest, how have you confirmed that the bar files contain exactly the same contents? |
|
| Back to top |
|
|
| jbanoop |
| Posted: Wed Nov 11, 2009 6:43 am Post subject: |
|
|
Chevalier
Joined: 17 Sep 2005
Posts: 401
Location: SC
|
Kirani,
#1 - No. It is not related to input data. The same messages that were stuck up when the flow experienced slowness was proccessed fast when the issue was rectified.
#2 The flow was running since June in the environment where the issue occured. No deployments/changes were done to the flow. The date was confirmed on the properties field of the deployed message flow.
WMBDEV1,
The "bad" bar file consistently creates the same problem. i.e. the issue can be recreated by deploying the "bad" bar file. There is only once instance of the flow deployed. We tried increasing the instances without any joy.
Yes, the two bar files contain exactly the same code.
Is there a possibility that the toolkit version has anything to do with the way in which the mapping node was built when being added to the BAR file ? |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Wed Nov 11, 2009 6:59 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| jbanoop wrote: |
Yes, the two bar files contain exactly the same code.
|
Sorry to keep banging on about this but i've learned to assume nothing on here  How did you confirm this?
For example, Did you look at the size of each file produced in the bar or did you use some other technique such as windif to compare the files within the bar?
If you deploy the bar file to another environment is the same behaviour experienced? |
|
| Back to top |
|
|
| jbanoop |
| Posted: Wed Nov 11, 2009 9:04 am Post subject: |
|
|
Chevalier
Joined: 17 Sep 2005
Posts: 401
Location: SC
|
There is only one version of the code in the clearcase stream. That is the code that has been used to create the "good" bar file as well as the "bad" one.
The mapping node where the flow seems to get lost in has been inspected and there is nothing but straightforward one-one mapping.
Yes, the deployment of the "bad" bar file to any environment has the "bad" effect and the deployment of the "good" bar file to any environment seems to have the "good" effect. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Nov 11, 2009 10:08 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Is there a discernable difference in the two bar files themselves? Size of the files, for example, size of the deployable resources with the bar files, etc. etc. etc...
You asked about the version of the Toolkit that was used to build the bar files. Is there a significant difference between the two? |
|
| Back to top |
|
|
|
|
