| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| WMQ5.3 WinXP / Queue Files - Research |
« View previous topic :: View next topic » |
| Author |
Message
|
| fjb_saper |
 Posted: Sun Aug 21, 2005 6:52 am Post subject: |
 |
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20766
Location: LI,NY
|
Have you read up on dspmqtrn/rsvmqtrn ?
Some of the reasons why to put uncommitted messages to queues certainly lies therein.
Think as well of long running transactions... with an external Transaction Manager
 |
|
| Back to top |
|
 |
| hopsala |
| Posted: Mon Aug 22, 2005 10:13 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Quote: |
| Have you read up on dspmqtrn/rsvmqtrn ? |
Ok, just re-read it, i'm still missing something here..
I am not talking about transactions that have been issued a prepare-commit but are still not commited, these I agree should be in the queue file (and logs). I am talking about *uncommited*, that is you issue a MQPUT under SYNCPOINT and this goes to the queue file.
If something goes wrong, mq rolls back the transaction and the application, upon trying to commit, will also get some error and thus backout (or simply not commit) changes in database or elsewhere. It's the same with external transaction managers.
There is no risk in getting rid of uncommited transactions, since the application also does not commit them, this is the whole idea. If you have a specific scenario in mind I haven't thought of, post it! 
| Quote: |
| Think as well of long running transactions... with an external Transaction Manager |
Agreed. As I said somewhere in my post (or was it another post? i'm getting old...) if the buffer fills or it is a long-running transaction all messages, commited and non-commited, persistent and non-persistent, should go to queue files. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Aug 22, 2005 10:28 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| hopsala wrote: |
I am not talking about transactions that have been issued a prepare-commit but are still not commited, these I agree should be in the queue file (and logs). I am talking about *uncommited*, that is you issue a MQPUT under SYNCPOINT and this goes to the queue file.
|
I guess I'm missing the distinction. Or, if I do understand the distinction, then how does a message get to the prepare-commit stage without first being uncommitted?
Regardless, if they're persistant messages, they need to end up in the queue file. So why stage them to the logs, and then have to recopy them to the q file?
If they're non-persistant messages... then they may never get written to the logs - even under syncpoint (as I understand it, at least). But they do still need to end up on the q file - unless it is determined that they do not need to get written to disk at all. So why stage them to the logs, when you don't need to put them in there in the first place, and then move them to the q file?
In the gentlest possible way, I'll suggest that the team at Hursley has been thinking about this stuff consistantly and almost exclusively for quite a long time now. So I have to think they've given it a good analysis, and have done what they've done for good reasons.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| hopsala |
| Posted: Mon Aug 22, 2005 12:50 pm Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| jefflowrey wrote: |
| how does a message get to the prepare-commit stage without first being uncommitted? |
Uncommited messages are message which are put to queue under syncpoint, but the application has not issued MQCMIT yet. prepare-commit messages are messages to which MQCMIT was issued but have still not completed the full commit cycle (i.e write to log, commit other resources if XA manager is involved etc etc).
You are right that msgs first have to be uncommited and then go to prepare-commit, but that's besides the point - these are logically two completely different states: uncommited means that if something goes wrong the application knows about it, and can thus react properly, prepare-commit means that the application already got RC=0 for her MQCMIT and is thus unable to resolve any errors.
Thus prepare-commit msgs should be written to log in case of mq crash immidiately after the appl got RC=0, in case the msgs are in prepare-commit, while uncommited msgs you can throw to the garbage. When mq restarts, without any uncommited msgs, the application, who got RC=error will retry putting and commiting.
I have a good familiarity with transactions in databases, mq, mq on z/os (which behaves as expected - not like this), CICS and IMS - there is a very specific algorithmic theory concerning transactions, and the books, plus every other product I know say - uncommited msgs are placed only in memory if possible.
| jefflowrey wrote: |
| Regardless, if they're persistant messages, they need to end up in the queue file. So why stage them to the logs, and then have to recopy them to the q file? |
That's not the way it works (in MQ or, for example, in DB2); there was a wonderful redbook with a precise explanation of all this, but it got withdrawn.
If messages are MQPUT to queue and then immediately someone MQGETs them, there is no reason to use queue files, that's not what their for; in such cases the msg is written to memory and to log, and upon MQGET is deleted from memory and marked as gotten in log file.
But say message was MQPUT to queue, written to memory and log and just then mq crashes, what then? Upon restart mq goes through the logs and queue files and sees that the queue file is missing that message, and then puts it to queue file.
This means that you only have 1 IO operation per message - write to log - not 2, which is a huge advantage; It means that runtime is much faster, but restart time is slower, which is exactly how mq behaves.
However, if memory buffers fill, or some message stays long in the queue then the msg is written to queue files. In z/OS queue files (or STGCLASS) are defined as used only for long-lived messages. This is not dissimilar to the way paging works in OSs.
| Quote: |
| If they're non-persistant messages... then they may never get written to the logs... |
I completely agree, non-persistent msgs should not be written to log, hope I didn't say something else by mistake somewhere...
| Quote: |
| In the gentlest possible way, I'll suggest that the team at Hursley has been thinking about this stuff consistantly and almost exclusively for quite a long time now. So I have to think they've given it a good analysis, and have done what they've done for good reasons. |
Hmm... I agree but disagree. Yes they have been thinking about it, but MQ is an old product, designed in an old, possibly outdated way of thinking, maybe now that the industry has learned so many valuable lessons they could have done better... ?
Also, as I told you, there is a very straight-forward way of working with transactions - and this is not it. It seems odd to me MQ should be different, and even odder to see that mq on z/OS should be different concerning such an important feature.
There may be a very good reason for all this, i'm not denying that, I just want to know what that reason is... |
|
| Back to top |
|
|
| hopsala |
| Posted: Sat Aug 27, 2005 4:49 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
No further ideas I see... I will try turning to IBM, see what they have to say.
If anyone has a good contact at IBM - a real expert, not some regular support chap - I would appreciate the reference. |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
