ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » Looking for information about MQ9 "Backup Queue Manager

Post new topic  Reply to topic
 Looking for information about MQ9 "Backup Queue Manager « View previous topic :: View next topic » 
Author Message
Lepiota
PostPosted: Tue Nov 22, 2022 6:17 am    Post subject: Looking for information about MQ9 "Backup Queue Manager Reply with quote

Newbie

Joined: 02 Jun 2015
Posts: 5

I may need a "Backup Queue Manager" soon (using RESET QMGR TYPE(ADVANCELOG) etc)

I've never used it before and have a couple of questions (below).
NB: For now, MIQM isn't possible.

I'd prefer a link to documentation or a presentation that has more information than the online "Configuring MQ" docs, but "backup queue manager" hasn't been a good search argument for DuckDuckGo

The questions:

1. Does this feature handle the "Channel Sequence Number Mismatch" issue ? I don't know any other way to deal with this, and we have a external partner that doesn't want to do a "RESET CHANNEL on their SDR.
1a. Edit: I just found qm.ini parameter "IgnoreSeqNumberMismatch".
Is this what Backup Queue Manager startup does to avoid forcing RESET CHANNEL on all the incoming SDRs?

2. How often can you copy the log file?

For (2) - I'm not trying to get it way down (a few minutes). I'm thinking perhaps every 30 or 60 minutes. FYI that will be well under half a (recovery) log file.
Back to top
View user's profile Send private message
Andyh
PostPosted: Wed Nov 23, 2022 7:49 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

Using the ADVANCELOG option results in the current log extent being completed and thus allows a consistent copy of the recovery log to be taken for a running queue manager. This results in the ability to take a consistent copy of the recovery logs at the point in time where the ADVANCELOG command was processed. You'd then expect a queue manager recovered from this data only to be inconsistent w.r.t other queue managers, and transaction mngers, which were not backed up at EXACTLY the same point in time (not possible),

It is then possible to recover the queue manager to this point in time, and so channel sequence numbers etc would all reflect that point in time.
IBM has made is easier in recent years to recover the queue manager from only a set of log extents. Depending on how recent your service level the process for reinstating the queue manager from only the log extents varies (both in operational complexity and in the time taken to rebuild).
Note that the log extents only allow queue manager objects to be recovered,
Some queue manager configuration (for example .ini files) needs to be backed up and restored independently. Obviously ALL non persistent messages are lost.

What is the situation you are actually trying to cope with ?
Might the new RAFT based MQ HA option be something you could use ?
Back to top
View user's profile Send private message
Lepiota
PostPosted: Thu Nov 24, 2022 12:58 am    Post subject: Reply with quote

Newbie

Joined: 02 Jun 2015
Posts: 5

We're doing a Unix to Windows migration, and trying to change as little as possible.

The setup of the original MQ environment is a bit eccentric. There are also some significant constraints in the IT environment, including some odd gaps in both infrastructure capability and in monitoring and automation.

The old environment runs on a single operating system, so we're migrating to a single OS.
The QM's are 100% throughput (no messages are ever supposed to sit in a QM for more than a few seconds), so despite the setup, MQ is (as usual) among the most reliable system/middleware components.

I'm not 100% happy with a single MQ environment of course. A QM is only "100% throughput" until a message consumer stops working, or loses its database

What I'd like is something simple that provides:
* The ability to keep the MQ available when e.g. Windows is patched (might not require "advancelog" etc)
* A reasonable chance that, if there's an unplanned buildup of production messages, we'll recover some of them

We don't have a proper monitoring/automation platform - if we do anything it will be with Windows Powershell.

I took a look at MQ "Native HA" for the first time after reading your post. It looks nice, and would be a good functional fit, but the Cloud requirement makes it impossible.
I don't have "disk mirroring" (yet), nor are MSCS, or the VMWare variant available.

A second identically configured QM on another Windows instance is possible though (& already built).

This led me to Channel Sequence Numbers and "Backup Queue Manager" via ADVANCELOG etc. It's not perfect of course, but it's not MQ's fault (or the customer's SysAdmins') that I don't have proper storage.
I couldn't find a lot of detailed information on either one, hence the original question. Finding "IgnoreSeqNumberMismatch" helped a lot, but I'd prefer not to learn the ins and outs of using "ADVANCELOG" on a live production system
Back to top
View user's profile Send private message
Andyh
PostPosted: Thu Nov 24, 2022 3:51 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

A backup queue manager managed using the advancelog command and the strmqm -r|-a options is really more of a DR solution than an HA solution and so doesn't seem like a very good fit for your problem.

The Native HA support is currently only available in some cloud environments, but there's nothing cloud specific about this support and I'd hope it will become more generally available at some point. Native HA is a bit like a more fully funtional "backup queue manager" where the logs are automatically replicated and replayed in real time.

Might a simple cluster of queue managers provide an interim solution until one of the missing pieces for a more complete HA environment is available ? Would the recently announced uniform cluster support help you to shift clients off the queue manager you want to work on ? It's a bit more complex than a simple singleton HA queue manager, because you need to consider the potential effect of messages being stranded on any inactive queue manager in the cluster, but I suspect it would be MUCH more mainstream than using backup queue managers (which I think almost nobody uses).
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Thu Nov 24, 2022 5:36 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

Lepiota,
What about Multi Instance Queue Managers? Might your environment meet the requirements (e.g. storage) to use this as a solution?

https://www.ibm.com/docs/en/ibm-mq/9.3?topic=configurations-multi-instance-queue-managers
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Nov 24, 2022 8:41 am    Post subject: Re: Looking for information about MQ9 "Backup Queue Man Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

Lepiota wrote:
NB: For now, MIQM isn't possible.

What other options have you pre-excluded?

Have you searched google? What were your results?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Nov 24, 2022 3:36 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

Have you considered using Windows MSCS HA cluster capability? This requires an active server and a passive server, with cluster disk(s) for the MQ qmgr data and logs, and a floating IP address / host name for MQ channels. It is fairly straight forward to set up.

If the active server needs to patched, it takes about 2 minutes or less to fail over (quiesce / start) the MQ qmgr to the other server. MQ channels should automatically retry and continue processing, with no sequence number errors. If the active server fails, the cluster resources can be configured to automatically fail over to the other server.

We use this extensively with Win 2012, 2016, 2019 Server.

MSCS supports multiple roles, so the cluster could be set up as active - active, with qmgr(s) running on both servers. This makes better use of hardware resources if they are physical servers. If one serve fails or needs an outage, the qmgr(s) can all temporarily run on the one server.

We also use this active - active cluster setup on AIX PowerHA UNIX for very critical and high volume qmgrs, but that's another story.
_________________
Glenn
Back to top
View user's profile Send private message
Lepiota
PostPosted: Fri Nov 25, 2022 2:41 am    Post subject: Reply with quote

Newbie

Joined: 02 Jun 2015
Posts: 5

Thanks for the responses. I'll try to cover all of them in one reply.

BTW I wrote some text covering why this situation is so strange, but removed it. It needed more information that I can share.

MIQM (good even without disk mirroring IMO) has been discussed and rejected.
MSCS and VMWare failover have been discussed and rejected.

IMO all three are probably technically possible (though I'm not 100% sure of MSCS and VMWare) but they're not available.

MQ Clustering was rejected as being too far from the system we're migrating from.
In hindsight, a "Uniform Cluster" would probably have worked well, and perhaps worth the fight for a non-minimal migration. But Clustering may have been "untestable" - there's an application-side issue that would have been very difficult to handle.

There would be other options if I had access to disk mirroring, but I don't.

I like the look of "Native HA" of course. That's interesting enough to get me to play around with it on the IBM Cloud (assuming that's possible for free). But it's not an option for the project.

Unless I've missed something, the best (least bad) option is to treat it as a DR-like situation.
It's not a 24/7 environment, so normal SysAdmin activities (e.g. patching) can be planned for quiet times. A bit old-fashioned but it works
But I also want something available for the "long-tail-risks".
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Nov 25, 2022 5:35 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

You seem to have rejected the usual and well-proven backup choices. This sounds like a computer science term paper. Thanks for playing.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Lepiota
PostPosted: Fri Nov 25, 2022 6:16 am    Post subject: Reply with quote

Newbie

Joined: 02 Jun 2015
Posts: 5

My customer rejected the usual and well-proven backup choices, for reasons that make sense in their particular situation.

It will be reworked in a few months as part of a larger project.
Meanwhile I'm looking for a reasonable path given the (unusual) constraints.
Back to top
View user's profile Send private message
exerk
PostPosted: Fri Nov 25, 2022 11:19 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

Lepiota wrote:
...It will be reworked in a few months as part of a larger project. Meanwhile I'm looking for a reasonable path given the (unusual) constraints.

If it's to be redone in a few months can't they wait?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Sat Nov 26, 2022 8:12 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

exerk wrote:
Lepiota wrote:
...It will be reworked in a few months as part of a larger project. Meanwhile I'm looking for a reasonable path given the (unusual) constraints.

If it's to be redone in a few months can't they wait?

In a few months, will the customer again summarily reject "... the usual and well-proven backup choices, for reasons that make sense in their particular situation."?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Sun Nov 27, 2022 3:14 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

Rejections are usually based around cost, time and effort, versus the risk profile of the business. eg. If it costs $200K to implement a DR solution, but the cost of a major disaster and recovery cost is only $100K and its only likely to happen once every 5 years, the management decision can be fairly obvious. In some cases a Disaster causes loss to other IT systems, customers and business partners, and long term damage to reputation and share price. This is harder to quantify.
_________________
Glenn
Back to top
View user's profile Send private message
bruce2359
PostPosted: Sun Nov 27, 2022 5:35 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.


It is the Triad of Cost, Quality and Time. You can only pick two.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Andyh
PostPosted: Mon Nov 28, 2022 6:14 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

Using an old style MQ "backup queue" manager properly is quite intensive in terms of the admin required. Each time a log extent is completed it has to be copied to the backup site and then replayed (with strmqm -r). It's then a customer responsibility to NEVER activate the backup queue manager without having assured that the active is never going to be restarted, and data loss (or gain) since the last log extent was copied is guaranteed. It's all VERY manual and therefore typically risky.
I'd be very surprised to find an environment where this was a better short term solution than MIQM, where the quality of service of the NFS file system would determine the level of availability (mirrors, synchronous/asynchronous replication etc).
At the end of the day the customer has to choose how much time, effort, and money to expend, balanced against the impact of a loss of availability or data integrity.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » General IBM MQ Support » Looking for information about MQ9 "Backup Queue Manager
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.