| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Log question replay messages |
« View previous topic :: View next topic » |
| Author |
Message
|
| Michael Dag |
 Posted: Fri Mar 26, 2004 5:16 am Post subject: |
 |
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| JasonE wrote: |
Every disk i/o hits performance. MQ write to the logs and queue files the minimum it has to in order to be able to guarantee that after a failure, no committed persistent messages are lost. As to how it does this - its magic  (ok - extremely complex!), and changes over time as more subtle optimizations arise. |
This is turning out as a very interesting topic!
Jason, do I understand from this discussion that in some cases, messages although persistent, never hit the disk?
I don't expect a full explanation of the magic, but can you give a or some simple examples which might make a helluva difference in performance when knowing some of the underlying rules?
a simple example: persistent messages travelling from one queuemanager to another IMHO hit the disk twice,
once in the transmission queue on the sending queuemanager and once on the localqueue on the receiving queuemanager.
How could this be prevented using persistent messaging and optimal system tuning?
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
 |
| JasonE |
| Posted: Fri Mar 26, 2004 5:29 am Post subject: |
|
|
Grand Master
Joined: 03 Nov 2003
Posts: 1220
Location: Hursley
|
Hmmm.... how to answer this! Both because its a complex area which I certainly dont fully understand, and because of internal issues.
As has been said before, messages may not get written to the **Q** file even if they are persistent, which is why things like cold starts are becoming more dangerous over time.
In your example, that is true and makes sense - after all either qmgr may need to recover the message (eg. machine dies before commit is received or after confirmation has occurred)
My limited understanding is that logging is unavoidable at the moment for (persistent) messages put under syncpoint. If you dont use syncpoint, then someone waiting may grab the message immediately without logging, but then you have no recovery. I think...
Also, always try to commit - The commit path is vastly faster than the backout path. The code is designed with the concept that you generally dont put something only to back it out again frequently.
As to trying to trace things by looking through the logs - thats a tricky one. If you are relying on dmpmqlog, it can only dump the active portions of logs of a shut down qmgr - Shutting down takes a checkpoint, and may have therefore removed the logging information you were actually interested in. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Mar 26, 2004 5:41 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| Frank Nellion wrote: |
| Jeff, I understand that they are not "actually" history logs in that sense, but as each message is date/time stamped, and if my code can handle that, surely I can store this somewhere and then manipulate the data to create a "history" file of my own, and if needs be model, simulate and compare? |
Right, you can. At least to the extent that the log files are a) understandable without proprietary IBM information, b) contain a complete snapshot of the information you really need, c) are "static" enough that you don't need to rewrite your entire program at every CSD...
Or you can also use a message exit or an API exit to log the same information. Or use a mirror queue to allow you to store stuff into a real data warehouse type archive. Or any of probably a thousand other ways to accomplish the same thing.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| markt |
| Posted: Fri Mar 26, 2004 5:41 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
One example of when there is no logging of a persistent message:
If an application puts a message outside syncpoint, and there is already a waiting getter, also specifying outside syncpoint, then the message is passed directly through with no logging. The MQPUT is not actually completed until after the MQGET, so the putting application doesn't get any return code (ie it has not been told that the message has been sent) until after the retrieval. So full guarantees can be kept of once-only delivery, maintaining the rules of persistence, and no forced I/O to disk.
There are more scenarios, some of which DO involve transactions, where there is no logging, but that's getting way too internal.
(And as Jason said, it does change over time. And may vary by platform too.) |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
