| Author |
Message
|
| cc |
 Posted: Wed May 28, 2008 3:30 am Post subject: WMB using Message Group Question |
 |
|
Novice
Joined: 28 May 2008
Posts: 19
|
I have a requirement to process groups of messages and combine them into one message to be sent downstream. I am looking for design ideas.
My first thought was to use an MQInput node with logical order and All messages available checked. Then if the MQMD Message Flag is not the last message write the message out to another queue. If the Message Flag is set to Last Message in Group, use an MQGet Node and a loop to get all of the messages in this group and build a structure in memory to combine all of the parts. Then propagate it downstream.
One issue I see with this method is that the original input queue can only have 1 Instance running, so there are Performance/Scalability concerns there.
Any thoughts? Alternative ideas? |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Wed May 28, 2008 3:53 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
there a Collector node in V6.1
if you are not in V6.1 use Aggregate nodes to do so
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| sridhsri |
| Posted: Mon Jul 21, 2008 7:04 pm Post subject: |
|
|
Master
Joined: 19 Jun 2008
Posts: 297
|
Its an interesting question. I have been playing with this a little now and here is what I think:
1) Collector node will not work - or at least I couldn't think of a way to use it. The node needs a quantity - how many msgs form the collective ? We don't know this. So we don't know when to stop looking for messages.
2) The MQInput node does not read the msg until the last message of the group has arrived in the queue. Perhaps there is a setting that will change this - but I couldn't find it.
So, knowing that - I would just say use SHARED variables or a Cache Node to store these messages in memory and join them all when you detect a last msg in the MQMD.MsgFlags (all this in a compute node). |
|
| Back to top |
|
|
| broker_new |
| Posted: Wed Jul 23, 2008 6:59 pm Post subject: |
|
|
Yatiri
Joined: 30 Nov 2006
Posts: 614
Location: Washington DC
|
| using a shared variable is a bad idea because there will be lot of messages coming in and configuring a cache node is too complex.I request the source system to include all the messages bind them into one big message and put it on a queue or let them insert all the messages into a temporary table and trigger the message flow using a cronjob and get all messages from the table and process. |
|
| Back to top |
|
|
| dcjs |
| Posted: Wed Jul 23, 2008 7:06 pm Post subject: |
|
|
Acolyte
Joined: 10 Nov 2006
Posts: 53
Location: IBM Bangalore
|
That would be a better idea but how do we know that the last record arrived and trigger the message flow using the cronjob.
I think the better way is the source system should club all the messages into a single big message and put on a queue.
_________________
TRY TRY UNTIL U SUCCEED |
|
| Back to top |
|
|
|
|
