| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Messages Drain too slowly |
« View previous topic :: View next topic » |
| Author |
Message
|
| bruce2359 |
 Posted: Sat Jan 28, 2012 8:29 pm Post subject: |
 |
|
Poobah
Joined: 05 Jan 2008
Posts: 9475
Location: US: west coast, almost. Otherwise, enroute.
|
Doesn't seem like much research went into your post. Did you read the APR and APG? Did you look at sample programs?
What else might Vitor have been suggesting?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
 |
| rekarm01 |
| Posted: Mon Jan 30, 2012 8:49 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| bruce2359 wrote: |
| Doesn't seem like much research went into your post. Did you read the APR and APG? Did you look at sample programs? |
bruce2359 has somehow misconstrued several hints meant to help him, as a request for his help. Oh, well. 
As for the APR, APG, and sample programs, he can do his own research. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jan 30, 2012 11:30 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20763
Location: LI,NY
|
| jkfriends wrote: |
BTW, I've solved this issue. Instead of doing the actual processing of the message in the same transaction as fetch, I simply shifted the control to a new thread. So in essence I spawn a new thread as soon as I have a good message to process. I can't believe the speed now. All 200 messages were consumed in 2 seconds (blazing fast).
Hope this will help someone who tried to solve the same problem.
Thanks everyone. You may close this thread. |
Not really. You have just shifted the onus on the client about transaction security and rollback and all that stuff that you now have to do yourself.
The right way to scale would have been to add more clients to process, i.e. more threads, not to pass off the message to a different thread.
What happens in case of a rollback? Does that mean that the thread now waits for all of the messages to be processed before it commits?
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jan 31, 2012 2:56 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
In addition to what fjb_saper has just said, you are not actually completing processing of these messages any sooner.
You are merely separating the time it takes to process these messages from the time it takes to get the messages off the queue.
So you get message 1, and then send it to your worker thread, and then get message 2, and send it to a new worker thread, and get message three and send it to a new worker thread, and then the first worker thread finishes.
Which, again, as fjb_saper says, means that if one of your worker threads encounters an error, you can't back out the message on the queue. Or if your code crashes, you have lost the contents of all of the messages that are still being processed.
This doesn't necessarily represent a problem, but it's a design consideration based on the business content of the messages. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed Feb 01, 2012 5:59 am Post subject: Re: Messages Drain too slowly |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
The OP might be long gone already, but someone else may find this useful.
If the .NET application uses a managed client connection or client bindings connection to a QMgr on a remote server, to GET/PUT messages under syncpoint, that in itself can significantly impact performance.
Using a server bindings connection to a QMgr on the local server can improve performance; this may require creating a QMgr on the local server if it doesn't already exist, and additional channels between QMgrs.
If the application requires some sort of transactionality, then reducing the number of commits between transactions can significantly improve performance. Instead of having the application issuing a commit after every transaction, it can issue a commit either after every N transactions, or after a time interval T elapses, (for suitable values of N and T). There are trade-offs with this approach, as the commit logic is a bit more complicated, uncommitted messages are not immediately available to receiving applications, and backing out may roll back more transactions than necessary, but the performance gain can make it worthwhile.
If the application really doesn't require any sort of transactionality, then using the NO_SYNCPOINT options for GET/PUT operations can improve performance. Using non-persistent messages improve the performance even further. Be warned that either of these approaches introduces the risk of losing messages, so make sure that's an acceptable risk.
If the application spends much of its time processing the transaction (outside of WMQ methods), there may be other ways to improve the processing performance, but the details would be application-specific. Profiling tools may help locate other performance bottlenecks.
If the application performance is still inadequate,
| fjb_saper wrote: |
| The right way to scale would [be] to add more clients to process, i.e. more threads ... |
When using multiple threads to handle transactions, each thread should have its own QMgr connection(s). When adding more threads or processes, it may also be useful to add more CPUs, or faster hardware. |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
