| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| how to calculate "-LS" from "-LP" |
« View previous topic :: View next topic » |
| Author |
Message
|
| Vitor |
 Posted: Thu Feb 05, 2009 1:44 am Post subject: |
 |
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| exerk wrote: |
| To be yet more specific, I read the post as an implication that using the rcrmqobj command would be used in the process. |
Well yes, if you've got one damaged queue on a production queue manager with 5000+ objects and thousands of users, bouncing it to invoke the automatic recovery might be a bit inconvienient and you would in this circumstance perform the recovery manually with the command you mention.
But if you've had a media failure and lost the queue files, it's quite convienient that the queue manager rebuilds itself from the logs at start up.
(Hence the recommendation to have logs & queues on separate devices in this kind of configuration)
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Thu Feb 05, 2009 7:03 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9475
Location: US: west coast, almost. Otherwise, enroute.
|
Media recovery is supported by control programs rcdmqimg and rcrmqobj, and only with linear logs. Media recovery is not performed at qmgr restart.
As part of qmgr restart, the qmgr will (if necessary) replay the logs if the it detects that there were logged events (get, put, alter, comit,...) after the last checkpoint. This is performed to bring the qmgr to its last known point of consistency. This is not media recovery.
Rcdmqimg is used to record an image of an object - like a queue definition and the messages contained in the queue - to the logs. If the object subsequently becomes damaged, you can use rcrmqobj to recover the last recorded image of the object.
I ponder the usefullness of media recovery because it uses the same logs. If the logs fail (damaged disk) the qmgr cannot restart. If my messages are 100meg each, recording them to the log after they've already been logged by mqput seems to be redundant and a waste of disk space.
If my objects are mission-critical, I'm likely going to use MS03 to save my object definitions, and write an application to copy long-lived messages to some other media.
If, like most mq apps, messages are put to a queue and consumed and comitted nearly instantly, there seems little practical value in recording the object image.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 05, 2009 7:06 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| Media recovery is supported by control programs rcdmqimg and rcrmqobj, and only with linear logs. Media recovery is not performed at qmgr restart. |
Beg to differ:
| Quote: |
If the queue manager object is damaged, the queue manager cannot start unless it can recover the object.If the queue manager is configured with a linear log, and thus supports media recovery, WebSphere® MQ automatically tries to re-create the queue manager object from its media images. If the log method selected does not support media recovery, you can either restore a backup of the queue manager or delete the queue manager.
|
http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/topic/com.ibm.mq.amqzag.doc/fa14660_.htm
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 05, 2009 7:19 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9475
Location: US: west coast, almost. Otherwise, enroute.
|
No argument from me on what IBM says, except that the sentence is not quite complete and accurate.
What's missing from the quoted sentence is the issue of whether and if the qmgr detects the damaged object at restart.
WMQ will only detect a damaged object when something touches it, like amqsbcg, mqget, display ql( ), AND the object is not in a buffer (forcing the qmgr to touch the disk where the damaged object lives.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 05, 2009 7:24 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| WMQ will only detect a damaged object when something touches it, like amqsbcg, mqget, display ql( ), AND the object is not in a buffer (forcing the qmgr to touch the disk where the damaged object lives. |
At start up, nothing is in a buffer, or will be in the process of being read into it (and hence touched!)
I'll leave an IBMer to discuss the amount of integrity checking the queue manager does at start up. I'll simply point out that:
a) I was attempting to correct your blanket assertion that automatic recovery does not happen
b) In the recommended scenario queue and logs are not on the same devces
c) Disc is cheap and wasted in bigger ways than this
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 05, 2009 7:35 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9475
Location: US: west coast, almost. Otherwise, enroute.
|
Here's a more accurate, moe complete, reference http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/index.jsp?topic=/com.ibm.mq.amqzag.doc/fa14640_.htm
Try this:
1) create a local queue BOB (on a qmgr with linear logs)
2) put some persistent messages to BOB
3) use rcdmqimg to record BOB
4) meander down the filepath to BOB
5) delete the little 'q' file inside BOB (this is the queue). It is now fatally damaged.
6) use amqsbcg to browse the messages. This will work because the messages are in buffer.
7) DIS QL(BOB). This will work - same reason.
 stop the qmgr
9) restart the qmgr.
10) redo steps 6 and 7. These should fail since the qmgr had to go to disk to find the damaged BOB and his messages.
11) recover the object with rcrmqobj
12) redo steps 6 and 7. These will work.
(it seems that the number 8 and a right paren yields the sun-glasses smiley)
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Feb 05, 2009 7:49 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| bruce2359 wrote: |
| If, like most mq apps, messages are put to a queue and consumed and comitted nearly instantly, there seems little practical value in recording the object image. |
Wellllll. I'm for circular logs as much as anyone, but last week we got a damaged object. 1st one in 5 years. Our SAN was real flakely last week, so I think that was the root cause.
The app didn't want to change the q name they were using, so we had to wait until the weekend to stop the QM (other apps use the QM), delete the damaged q file, replace it with a good q file, restart the QM and delete/recreate the q in runmqsc. If we had linear logs we could have just run the command to recreate the damamged object.
But I was able to jam an upgrade to MQ 6.0.2.5 (from 6.0.2.1) in 3 days time saying IBM recommended we do that (they did in the PMR), so some good came out of it.
I still don't understand why IBM can't add the ability to recreate damaged objects from circular logs. Just let the QM with circular logs take media images every hour or so automatically and store it in the circular logs. What's the big deal? (says the guy who has no clue what's really happening under the covers)
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 05, 2009 7:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Now you try the same, omitting step 3.
I'll conceed that the queue manager checking is less complete than I thought - I tried the same trick with SYSTEM.AUTH.DATA.QUEUE and got the messages back. I also checked the queue manager I was using for reference and there are user processes reading the queue which start with the queue manager, clearly "touching" them.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Feb 05, 2009 7:52 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| PeterPotkay wrote: |
| I still don't understand why IBM can't add the ability to recreate damaged objects from circular logs. Just let the QM with circular logs take media images every hour or so automatically and store it in the circular logs. What's the big deal? (says the guy who has no clue what's really happening under the covers) |
There's been some explanation here before, that's likely as good as you're going to get. I think in the requirements thread? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 05, 2009 8:23 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9475
Location: US: west coast, almost. Otherwise, enroute.
|
Circular vs. linear. Less filling vs. tastes great.
I usually don't take a strong stand on this (I'm a consultant). Each offers value.
Circular made sense when disk space was expensive and in short supply - not so today.
Linear make sense if there's some reason to keep (archive) log segments for whatever reason, or you want the opportunity for media recovery, however limited and misunderstood.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Feb 05, 2009 2:04 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
Perhaps there is now a case to be made for tertiary logs on distributed platforms, e.g. -lt, that, for circular logging only, will hold images of objects, written at the same time as a check point in the primary/secondary logs.
To elaborate: if object images were kept in circular logs, and a small number of primary/secondary logs are initially defined, there might be insufficient room within the logs to record all the information without having to cut additional logs, and maybe run out of logs before a wrap-around reaches the first log (unlikely I know given the size that logs can be now). Tertiary logs would hold only object images and the information for the -lp and -ls logs to reference them. Therefore, at queue manager creation time, an additional argument would be passed (no argument = no tertiaries) to create them, with a default size and number equal to the -lp argument, unless otherwise stated - after all chaps, and ladies, we as admins know how many logs, and their size, we need don't we? These logs would also wrap as their contents could be over-written at checkpoint, therefore media recovery would only be for one given point in time, i.e. damaged object recovery and not 'old' message replay.
Now, at this moment, this seems like a really good idea, but then I have been in the bar and in the cold light of day this may be a right crock, so I am happy to be shot down in flames - on the other hand, it may be a candidate for both the current challenge, or the wish list!
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Feb 05, 2009 6:44 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20763
Location: LI,NY
|
Well I like the idea but then the problem is that logs can roll over so fast in times of heavy usage... So what do you do when the last good image of the queue is 15 / 20 logs away and you have only defined 5 primaries and 2 secondaries? Having circular logs are you ready to overwrite the good image of the queue with a bad one?
There are more problems here than it seems.
I trust the experts and use linear logs. Deleted a queue file once to test it and the file was restored nice and the queue had the messages again... mind you only the persistent ones...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Feb 05, 2009 8:41 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| fjb_saper wrote: |
| So what do you do when the last good image of the queue is 15 / 20 logs away and you have only defined 5 primaries and 2 secondaries? |
If you defined only 7 logs (5 + 2), there is no way your images could be 15-20 logs away. If you fill the logs, today, or in the future dream world where the circular logs have media images, then the behaviour is the same - the app gets a forced rollback and the QM moves on.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Feb 06, 2009 4:10 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20763
Location: LI,NY
|
| PeterPotkay wrote: |
| fjb_saper wrote: |
| So what do you do when the last good image of the queue is 15 / 20 logs away and you have only defined 5 primaries and 2 secondaries? |
If you defined only 7 logs (5 + 2), there is no way your images could be 15-20 logs away. If you fill the logs, today, or in the future dream world where the circular logs have media images, then the behaviour is the same - the app gets a forced rollback and the QM moves on. |
And how pray does the QM move on when you cannot write the next checkpoint because the queue has gone bad and you would be overwriting the last check point where the queue was good?
Does this call for a qmgr stop?
I think we have hit here one of the differences between qmgr restart and media recovery logs...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
