| Author |
Message
|
| George Carey |
 Posted: Wed May 07, 2008 3:40 pm Post subject: Snap mirroring and MQ |
 |
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
Is there a command or way to do the following:
Step 1.)Checkpoint MQ
by which I mean suspend all new activity.. (just like the quiesce function on shutdown) and complete current activity and flush all cached messages and buffers to disk, then return to command line without ending MQ.
Step 2.) to allow say a snap dump of disk volume blocks for logs and queues to be taken and
Step 3.)then to resume activity for MQ ...
This would be similar to putting a database like Oracle in hot backup mode with a command take a snap dump and then resuming the database.
Is this doable with any IBM or vendor product ?
TIA
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding") |
|
| Back to top |
|
 |
| ashoon |
| Posted: Wed May 07, 2008 6:59 pm Post subject: Re: Snap mirroring and MQ |
|
|
Master
Joined: 26 Oct 2004
Posts: 235
|
I can't think of anyway to do so BUT I've always wondered why people would want to do this considering that MQ messages (for the most part) have a lifespan of max 1 minute since it's transient data...
_________________
IBM Certified - SOA Solution Designer & WebSphere Datapower SOA Appliances |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed May 07, 2008 8:06 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
rcdmqimg
?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed May 07, 2008 8:14 pm Post subject: |
|
|
Jedi
Joined: 25 Mar 2003
Posts: 2498
Location: Melbourne, Australia
|
MQ is not a database, it is a store-and-forward messaging facility. In theory, all queues will spend most of their time near empty. It has database-like coordination and recovery features (syncpoint and logging), but what are good reasons for taking a point-in-time snapshot of all queued data in a currently running queue manager?
_________________
Glenn |
|
| Back to top |
|
|
| George Carey |
| Posted: Thu May 08, 2008 9:37 am Post subject: remote site synchronization |
|
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
Agreed don't won't to use MQ as DB but ...
May need to use MQ to hold work request messages for indeterminate time before persisting into real DB table such as Oracle's or DB2's ...
So possible thousands of messages sitting in work request queue(s) ....
Need is to replicate MQ Queue content in synchrony with DB content to a remote site ... and maintain this synchrony.
Potential of redoing same work has issues ... and
of course losing work has issues ...
but if MQ and DB in synchronized state at a point in time, issues may be able to be dealt with ...
So say 80 work request sitting on MQ Q and 20 in DB and site A goes away
don't want replicated data at:
site B to show 80 in queue and 10 in DB or
site B to show 70 in queue and 20 in DB but
if site B shows 90 in queue and 10 in DB this is OK,
for example.
... a point in time synchronized recovery or ideally a CDP concept
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding") |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu May 08, 2008 9:43 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
backup queue manager.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu May 08, 2008 10:02 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
| jefflowrey wrote: |
| backup queue manager. |
but that only ships the linear log to the backup QM as the linear log on the active QM rolls over. If your linear log is big and/or the message volume is relatively low, it can be quite a while before that happens, meaning your backup QM can be hours (days?) behind.
I can see how George's idea might be useful. If the snap could happen quickly, VERY quickly, and there was a reliable way of insuring the snap was 100% consistent in all aspects, then it would be a good way of acheiving this. But think about all the inflightr stuff happening on a busy QM with lots of UOWs going on. Getting that all coordinatred and paused in a second or two is a tall order.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu May 08, 2008 2:20 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20697
Location: LI,NY
|
pub/sub... Have both the normal DB and Backup DB subscribe to the messages...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu May 08, 2008 2:23 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
How do you insure the publications are consumed at the same rate (or at all) at the DR site as your live site?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu May 08, 2008 2:30 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20697
Location: LI,NY
|
| PeterPotkay wrote: |
| How do you insure the publications are consumed at the same rate (or at all) at the DR site as your live site? |
you need to have the consumption scalable.
When the queue is empty you should be uptodate
if you checked that there is nothing stuck on the way...(all channels running etc..., dlq's empty ...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu May 08, 2008 3:15 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
What I meant was, who consumes the publications in your DR environment while production is running live in the real environment?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| George Carey |
| Posted: Thu May 08, 2008 8:44 pm Post subject: |
|
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
| Quote: |
| If the snap could happen quickly, VERY quickly, and there was a reliable way of insuring the snap was 100% consistent in all aspects, |
Yes that is my concern if it could be done ... really what I am talking about is basically what is being done already by MQ when one does the endmqm command ... the queue manager goes into quiescing mode flushes buffer to disks, etc. then kills all the mq processes ... only my idea is don't kill the processes ... just wait until a 'resume' mqm command is given to go out of quiesing mode and back into active mode, like a strmqm.
I got to believe this would be easy for IBM MQ developers to implement. An as far as UOW.. if they were forced to fail, they roll back don't they and as long as they could be re-issued successfully, I mean what's UOW processing for otherwise. What happens with an endmqm -i I forget ?
And yes the issue is not getting messages to DR site but getting site A current queue image to site B .. and also DB image in synchrony.
Also on devices like NetApp one can specify the exact volumes to monitor for disk block image changes for snap dump, the volumes would be the volumes allocated for MQ Queues and logs, and of course appropriate volumes for the DB.
Pause MQ, Pause DB, snap volumes (MQ & DB), resume MQ resume DB ! Have background process looking for snap files to mirror to DR site.
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Last edited by George Carey on Fri May 09, 2008 7:17 am; edited 1 time in total |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Thu May 08, 2008 11:16 pm Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)
|
|
| Back to top |
|
|
| George Carey |
| Posted: Fri May 09, 2008 7:01 am Post subject: CPD |
|
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
Already talking with them, they don't quite have what I need, the ability as I state above, but they have a replication product that I might be able to use, ... AND
OH by the way ... They can recover a damaged queue from circular logs ... it just will only have the default queue attributes (wow big restriction !!!).
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding") |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri May 09, 2008 7:21 am Post subject: Re: CPD |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
| George Carey wrote: |
| OH by the way ... They can recover a damaged queue from circular logs ... it just will only have the default queue attributes (wow big restriction !!!). |
Is this feature detailed in their doc or you only found this out in talking with them?
Will the persistent messages be there?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
