| Author |
Message
|
| lchiu7 |
 Posted: Thu Jul 07, 2022 5:12 pm Post subject: Streaming messages going to one queue to another one - query |
 |
|
Novice
Joined: 27 Apr 2022
Posts: 10
|
We have a situation where we receive inbound messages via MQ and forward them to another external queue for processing. But we want to stream that set of messages to an internal queue for other purposes. We would prefer not to update the applicaiton that is receiving the messages and writing them to MQ.
Looking around I see where is a feature called MQ Streaming Queues which seems like it could do the job
https://www.ibm.com/docs/en/ibm-mq/9.2?topic=scenarios-streaming-queues
Reading it briefly it seems that the service runs on the target queue on the external queue manager where the messagea are doing. That might not suit us as the messages are now gone from us. Has anybody used this facility and if so, is that the correct assumption on how it's implemented? Thanks |
|
| Back to top |
|
 |
| abhi_thri |
| Posted: Fri Jul 08, 2022 12:40 am Post subject: Re: Streaming messages going to one queue to another one - q |
|
|
Knight
Joined: 17 Jul 2017
Posts: 516
Location: UK
|
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jul 08, 2022 5:19 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Fri Jul 08, 2022 2:14 pm Post subject: Re: Streaming messages going to one queue to another one - q |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3253
Location: London, ON Canada
|
| lchiu7 wrote: |
| We have a situation where we receive inbound messages via MQ and forward them to another external queue for processing. But we want to stream that set of messages to an internal queue for other purposes. We would prefer not to update the applicaiton that is receiving the messages and writing them to MQ. |
You should have a look at Capitalware's MQ Message Replication (MQMR). Its whole purpose is to do exactly that. MQMR is not a separate application but rather an MQ API Exit. It sees everything and MQMR can clone (copy) messages from any queue and put an exact copy of the message (header & payload) to up to 100 target queues. A target queue can be of any type of queue: local, remote, alias, cluster.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Jul 08, 2022 2:14 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
Streaming Queues can do this for you the easiest. You would need to upgrade your queue manager to an MQ version that supports it.
You said "we receive inbound messages via MQ". It is on that queue manager on the receiving queue where you would configure the stream attribute to produce copies "to an internal queue for other purposes."
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Fri Jul 08, 2022 2:18 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3253
Location: London, ON Canada
|
For streaming queues, you need IBM MQ v9.3 and the setup is 1 to 1 (not 1 to many). And you can NOT stream a streaming queue.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| lchiu7 |
| Posted: Fri Jul 08, 2022 6:32 pm Post subject: |
|
|
Novice
Joined: 27 Apr 2022
Posts: 10
|
| PeterPotkay wrote: |
Streaming Queues can do this for you the easiest. You would need to upgrade your queue manager to an MQ version that supports it.
You said "we receive inbound messages via MQ". It is on that queue manager on the receiving queue where you would configure the stream attribute to produce copies "to an internal queue for other purposes." |
I asked around and obtained more information. Messages arrive via MQ externally and after a small amount of processing are placed on a queue on an internal QM. That QM apparently has some settings (I am hedging terminology here since I am not a MQ person but said I would ask the question) to then send the messages to another queue on an external. From reading the replies it seems I could stream the messages to another queue from the local queue.
Last edited by lchiu7 on Fri Jul 08, 2022 6:40 pm; edited 1 time in total |
|
| Back to top |
|
|
| lchiu7 |
| Posted: Fri Jul 08, 2022 6:37 pm Post subject: |
|
|
Novice
Joined: 27 Apr 2022
Posts: 10
|
| RogerLacroix wrote: |
For streaming queues, you need IBM MQ v9.3 and the setup is 1 to 1 (not 1 to many). And you can NOT stream a streaming queue.
Regards,
Roger Lacroix
Capitalware Inc. |
I think we at the right level and it's 1:1 we want. Thanks |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Jul 09, 2022 6:05 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
| lchiu7 wrote: |
| I think we at the right level and it's 1:1 we want. Thanks |
I apparently misunderstood your requirement as a 1:2 - inbound messages replicated to two output queues.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| lchiu7 |
| Posted: Sat Jul 09, 2022 3:06 pm Post subject: |
|
|
Novice
Joined: 27 Apr 2022
Posts: 10
|
| bruce2359 wrote: |
| lchiu7 wrote: |
| I think we at the right level and it's 1:1 we want. Thanks |
I apparently misunderstood your requirement as a 1:2 - inbound messages replicated to two output queues. |
I am confused now. I want to be able to write to one queue and have the QM then replicate the messages to another one. Isn't that what streaming queues allows you to do? Otherwise what does streaming queues do for you? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Jul 09, 2022 4:03 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
| lchiu7 wrote: |
| bruce2359 wrote: |
| lchiu7 wrote: |
| I think we at the right level and it's 1:1 we want. Thanks |
I apparently misunderstood your requirement as a 1:2 - inbound messages replicated to two output queues. |
I am confused now. I want to be able to write to one queue and have the QM then replicate the messages to another one. Isn't that what streaming queues allows you to do? Otherwise what does streaming queues do for you? |
There are several solutions to the replication/duplication of messages.
You used the term 'stream' in your OP. If you searched google using 'MQ stream,' likely you landed on new-to-MQ 9.2's new stream capabilities. If you merely wanted to replicate messages to a 2nd queue as they are put, then use search terms 'replicate' or 'duplicate'.
Read here: https://www.ibm.com/docs/en/ibm-mq/9.2?topic=scenarios-streaming-queues
The pub/sub solution I offered earlier is well-established solution to your requirement. Well-established here means 'not new'.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| lchiu7 |
| Posted: Sun Jul 10, 2022 6:54 pm Post subject: |
|
|
Novice
Joined: 27 Apr 2022
Posts: 10
|
| bruce2359 wrote: |
| lchiu7 wrote: |
| bruce2359 wrote: |
| lchiu7 wrote: |
| I think we at the right level and it's 1:1 we want. Thanks |
I apparently misunderstood your requirement as a 1:2 - inbound messages replicated to two output queues. |
I am confused now. I want to be able to write to one queue and have the QM then replicate the messages to another one. Isn't that what streaming queues allows you to do? Otherwise what does streaming queues do for you? |
There are several solutions to the replication/duplication of messages.
You used the term 'stream' in your OP. If you searched google using 'MQ stream,' likely you landed on new-to-MQ 9.2's new stream capabilities. If you merely wanted to replicate messages to a 2nd queue as they are put, then use search terms 'replicate' or 'duplicate'.
Read here: https://www.ibm.com/docs/en/ibm-mq/9.2?topic=scenarios-streaming-queues
The pub/sub solution I offered earlier is well-established solution to your requirement. Well-established here means 'not new'. |
This betrays my lack of knowledge in the area of MQ. The users want messages to go to another queue for backup purposes. The developers don't want to change their code (this is Cobol mainframe code). So I did some Googling and came across the streaming option which appears can be done in MQ without code changes. But the MQ admin team said (without much detail) it also could be done with publish/subscriibe which you mentioned. Found some pages on that but it's not clear to me if that can be done without code changes. Anyway if we have the right level of MQ streaming seems to be the way to go. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Jul 10, 2022 9:09 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
Neither solution suggested here requires application source code changes - only MQ object definitions. Try both in test.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Mon Jul 11, 2022 3:32 pm Post subject: |
|
|
Jedi
Joined: 25 Mar 2003
Posts: 2494
Location: Melbourne, Australia
|
| lchiu7 wrote: |
| This betrays my lack of knowledge in the area of MQ. The users want messages to go to another queue for backup purposes. The developers don't want to change their code (this is Cobol mainframe code). So I did some Googling and came across the streaming option which appears can be done in MQ without code changes. But the MQ admin team said (without much detail) it also could be done with publish/subscriibe which you mentioned. Found some pages on that but it's not clear to me if that can be done without code changes. Anyway if we have the right level of MQ streaming seems to be the way to go. |
The pub/sub technique does not require code changes, but the queue name where the producer app puts messages will need to be deleted and replaced with a QALIAS that has targtype(topic).
If the existing consumer app gets messages from the same queue name and same qmgr as the producer, you'll need to set up a new local queue name for the consumer (it will be the destination of an administrative SUB object). If its on another qmgr, the existing name can be kept.
_________________
Glenn |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jul 12, 2022 7:18 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
| lchiu7 wrote: |
| This betrays my lack of knowledge in the area of MQ. |
I recommend that you enroll in a basic MQ System Administration course. Search google for 'ibm mq basic system administration training'
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
