Author |
Message
|
SpokeU |
Posted: Tue Jul 18, 2017 12:28 am Post subject: Sequence number between flows |
|
|
Newbie
Joined: 06 Oct 2016 Posts: 4
|
Hi All,
I have a few flows that are communicating through queues.
The first one RecieverFlow accepts initial messages and sends those to ProcessorFlow for processing.
After processing happens messages are sent to SendFlow to send it to third parties.
But the requirement is that SendFlow must send messages in the same sequence that RecieverFlow recieved it.
So if I want a paralel processing of messages in ProcessorFlow I need to stamp some sequence number to it in RecieverFlow with Sequence node in order to synchronize them in SendFlow through Resequence node.
The question is where should I put that sequence number of a message. I don't want to pass it in message body as services has already defined xsd schemas and I can't add some sequence number to message payload.
I've tried to set it to MQMD MsgSeqNumber header by setting "Path to store sequence number" in Sequence node to "$OutputRoot/MQMD/MsgSeqNumber". But when ProcessorFlow recieve a message MsgSeqNumber always 1 so I'm not sure if this is the right place.
Can anyone point me where should I put this sequence number to pass it with message.
I'm using 9.0.0.5 message broker.
Thanks |
|
Back to top |
|
|
Vitor |
Posted: Tue Jul 18, 2017 4:32 am Post subject: Re: Sequence number between flows |
|
|
Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
SpokeU wrote: |
But the requirement is that SendFlow must send messages in the same sequence that RecieverFlow recieved it. |
Congratulations - you join the long line of people who discover why message affinity is a bad design decision.
SpokeU wrote: |
I don't want to pass it in message body as services has already defined xsd schemas and I can't add some sequence number to message payload. |
Yet this might be your only viable option.
SpokeU wrote: |
I've tried to set it to MQMD MsgSeqNumber header by setting "Path to store sequence number" in Sequence node to "$OutputRoot/MQMD/MsgSeqNumber". But when ProcessorFlow recieve a message MsgSeqNumber always 1 so I'm not sure if this is the right place. |
It's not. That field in the MQMD is used by MQ, which is why it's being reset to 1.
SpokeU wrote: |
Can anyone point me where should I put this sequence number to pass it with message. |
Anywhere in the message. _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
|
SpokeU |
Posted: Tue Jul 18, 2017 4:53 am Post subject: |
|
|
Newbie
Joined: 06 Oct 2016 Posts: 4
|
Thanks Vitor,
So just to summarize -
The only way to pass message sequence number is to pass it in message body. Correct? |
|
Back to top |
|
|
fjb_saper |
Posted: Tue Jul 18, 2017 5:04 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003 Posts: 20729 Location: LI,NY
|
SpokeU wrote: |
Thanks Vitor,
So just to summarize -
The only way to pass message sequence number is to pass it in message body. Correct? |
No. If using MQ you could also pass it as a message property _________________ MQ & Broker admin |
|
Back to top |
|
|
Vitor |
Posted: Tue Jul 18, 2017 5:17 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
fjb_saper wrote: |
SpokeU wrote: |
Thanks Vitor,
So just to summarize -
The only way to pass message sequence number is to pass it in message body. Correct? |
No. If using MQ you could also pass it as a message property |
Which in IIB terms is expressed as a RFH2 header for the record.
It's a good way of not adding it to the message body; my worthy associate produces a worthy idea.
I still maintain that message affinity is a bad practice as, even once you've fixed this sequence number problem, you've deliberately built a bottleneck. _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
|
inMo |
Posted: Thu Jul 20, 2017 9:51 am Post subject: |
|
|
Master
Joined: 27 Jun 2009 Posts: 216 Location: NY
|
Quote: |
So if I want a paralel processing of messages in ProcessorFlow |
Why do you want parallel processing in the processing flow? Does it solve a problem? |
|
Back to top |
|
|
SpokeU |
Posted: Tue Jul 25, 2017 12:00 am Post subject: |
|
|
Newbie
Joined: 06 Oct 2016 Posts: 4
|
Quote: |
Why do you want parallel processing in the processing flow? Does it solve a problem? |
Yes, because the problem is with huge load that needs to be balanced. I understand that there will be a bottleneck on send flow as its a sync point. But the major point here is a processing stage as it gets a lot of resources. |
|
Back to top |
|
|
mqjeff |
Posted: Tue Jul 25, 2017 5:19 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
Is there already some kind of sequence number in the message, or some kind of data that can be used as a squence number? (timestamp, or etc) ?
If not, you can set message properties (not fields in the mqmd) that contain a number you need...
But if you can't trust the order of messages coming in - for example if your input flow has more than one instance - then a resequence node doesn't help. Without use of a shared variable or the global cache in an atomic block you could easily get more than one message with the same sequence number.
Best bet is to find a unique id in the message itself that indicates the proper sequence. _________________ chmod -R ugo-wx / |
|
Back to top |
|
|
|