| Author |
Message
|
| jugadu |
 Posted: Wed Aug 26, 2009 3:46 am Post subject: Reducing process time |
 |
|
Apprentice
Joined: 04 Dec 2007
Posts: 30
|
Hi veterans..
Here is a problem I am facing in the WMB flow..
MQinput is acting like a flow triggerer. When a message is put on the input Q, the flow picks up all of the data in Oracle database table, creates a cobol copy book out of it. This is done in a compute node.The file is being send to remote ftp box through fileoutput Node.
For 40K rows in database, it is taking about 1.5 hrs to complete the job which is bothoring Client. File size for 40K orders is abut 15.5MB.
How to go about solving this scenario?
From the DB we are taking whole data into Environment VAriable root and then creating CCB from it through message sets. Previously the Env VAriable was not getting cleared in code after each row was read. I did this change after facing probs with process time. To my wonder, this did not have even a slightest change in Process time.
It is established that DB interaction is not taking time. File creation is only taking a huge share of time.
Plz suggest... |
|
| Back to top |
|
 |
| WMBDEV1 |
| Posted: Wed Aug 26, 2009 4:17 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| Quote: |
| File creation is only taking a huge share of time |
So its taking a long time to FTP the file over the network? Maybe chat to your network guys?
Does it take a long time when doing FTP outside of the broker? |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Wed Aug 26, 2009 4:18 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| Also, can an admin move this to the WMB section? |
|
| Back to top |
|
|
| jugadu |
| Posted: Wed Aug 26, 2009 4:45 am Post subject: |
|
|
Apprentice
Joined: 04 Dec 2007
Posts: 30
|
| Ftp is not taking time. when I do it manually, it takes 30 seconds. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 26, 2009 4:58 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Take a debug level user trace of the flow in action.
See from that where all the time is being taken. My first guess is that you are using indexes instead of references to access Environment. |
|
| Back to top |
|
|
| jugadu |
| Posted: Wed Aug 26, 2009 6:41 am Post subject: |
|
|
Apprentice
Joined: 04 Dec 2007
Posts: 30
|
| Yes you are very much correct..Does this effect the process time? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 26, 2009 6:55 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| jugadu wrote: |
| Yes you are very much correct..Does this effect the process time? |
Yes.
VERY MUCH.
Every time you access a field by index, for example [n], Broker must navigate through the message tree from FIRST CHILD and count until it reaches n. So it must count to get to n. And then when you ask for n+1 it must count to n and then count one more.
See the Performance Report SupportPacs and SupportPac IP04 http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24006518&loc=en_US&cs=utf-8&lang=en |
|
| Back to top |
|
|
| jugadu |
| Posted: Fri Aug 28, 2009 8:01 am Post subject: |
|
|
Apprentice
Joined: 04 Dec 2007
Posts: 30
|
Thanks so muchhhhhhh.
Process time got reduced to 85%.  |
|
| Back to top |
|
|
| jugadu |
| Posted: Wed Sep 23, 2009 4:11 am Post subject: |
|
|
Apprentice
Joined: 04 Dec 2007
Posts: 30
|
Thanks a lot for you help guys!! really appreciate!!
Same scenario
I compute a part of the message and keep sending it to the IN termial of Fileoutput Node. This is under a while loop.
When I am done, I send a flag to Finish File terminal.
Now when I compare Process times, its not linear for no. of Orders given.
One order = One part
Orders | ProcessTime
20,000 | 83 seconds
40,000 | 6 minutes
60,000 | 17 minutes 30 seconds
What is bothering me is "why not 8-9 minutes for 60,000 orders(parts)?"
Why there is no linear relation between no. of orders and [b]process time
Please help.. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Sep 23, 2009 5:16 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I'm more bothered about your mental arithmetic 
(60,000 / 20,000) * (83 seconds) = 249 seconds, or about 4 minutes.
However, you're right to be bothered about the non-linear scaling. I don't have any specific advice, but performance tuning should always be done using evidence about where the time is being taken. So your first steps should be aimed at finding out which part of the process is the non-linear part. |
|
| Back to top |
|
|
|
|
