ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ API Support » Can You Intentionally Delay a Message's Availabilty?

Post new topic  Reply to topic Goto page 1, 2  Next
 Can You Intentionally Delay a Message's Availabilty? « View previous topic :: View next topic » 
Author Message
queuetip
PostPosted: Tue Mar 24, 2009 12:19 pm    Post subject: Can You Intentionally Delay a Message's Availabilty? Reply with quote

Acolyte

Joined: 03 Feb 2005
Posts: 67

Hi!

I have an application requirement to delay a message's availability for a configurable amount of time (e.g. after a msg is put on queue, wait 15 seconds before processing). Messages are less than 1K in size.

I was wondering if MQSeries V7 had any built-in administrative/ application interface options that could accomplish this delay for for us? I didn't see any. Looking for this forum's help for alternatives & recommendation.


Here are the 2 techniques I'm thinking about:

1) Creator application puts message on the data FIFO queue...MQSeries triggers consuming application on FIRST...consuming application opens data queue and gets message off immediately...add the delay time (e.g. 15 seconds) to the PutApplicationDateTime field...retrieve the system current datetimestamp.

If the current time is less than calculated time, application issues a wait for the difference before continue processing; otherwise continue processing.

2) Creator application builds message with delay time (e.g. 15 secs) in the expiry field, specifiy report option of MQRO_EXPIRATION_WITH_FULL_DATA & specify a ReplyToQ in the MQMD. Creator puts message on FIFO queue...MQSeries triggers consuming application on FIRST...consuming application opens both data queue (only to activate discard process) and ReplyToQ....consuming application gets messages on ReplyToQ with delay time as wait interval (knowing that delay has expired).

Any other alternative techniques? Recommendations?

Thank you!
Mike
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Tue Mar 24, 2009 12:43 pm    Post subject: Re: Can You Intentionally Delay a Message's Availabilty? Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3258
Location: London, ON Canada

queuetip wrote:
I have an application requirement to delay a message's availability for a configurable amount of time (e.g. after a msg is put on queue, wait 15 seconds before processing). Messages are less than 1K in size.


Why? This is a bad idea that will require a really bad design to be implemented.

Please tell your boss/management that it is best to simply consume the messages as they arrive.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
mqjeff
PostPosted: Tue Mar 24, 2009 12:45 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

It's not clear at all why it's a good idea to delay a message after it's been sent, rather than delay sending it in the first place.

No, MQ doesn't have a function for this because it's a really wasteful idea.

You can easily do this with Mesage Broker, though, and the Timeout nodes. If you don't see the need for a full enterprise license, you can purchase the very reasonably priced Starter Edition.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Mar 24, 2009 12:48 pm    Post subject: Re: Can You Intentionally Delay a Message's Availabilty? Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

queuetip wrote:
Recommendations?


Ask whoever gave you the requirement if they're serious?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
queuetip
PostPosted: Tue Mar 24, 2009 1:36 pm    Post subject: Reply with quote

Acolyte

Joined: 03 Feb 2005
Posts: 67

C'mon guys...in general, I agree with you but if you are truly experienced, you'd be a little more humble & realize there are always some unique processing requirements!


Besides if having message delay is so wrong, maybe we should ask why Oracle AQ extended their product with this option?


Seriously, I really would like your help...you guys have helped me out many times before. I will elaborate to help you better understand my scenario a little more...

The reason the application creating the message can't wait is that it is a real-time transaction and a clerk is expecting a sub-second response (we have SLAs)! On the back-end, we are coordinating data updates with a 3rd party system (outside the MQSeries unit of work). These messages are used to drive a background process that reads databases to build data messages that are fed near-real time to this other system. Since the same message can be produced by multiple clerks working independently and concurrently, we have determined we can eliminate creating a significant amount of "duplicate" data messages if we simply wait a few seconds due to the nature of the activity. We needed to establish a boundary for what is classified as duplicate vs. changes for the sake of the 3rd party system.

So this is why I am interested in possible delay techniques. Hoping this helps support our need for the intentional delay.

Again, thanks for understanding about the business environment & requirements. I look forward to any ideas you may have to help me accomplish the delay.

Thanks!
Mike
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Mar 24, 2009 2:01 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9442
Location: US: west coast, almost. Otherwise, enroute.

Quote:
(e.g. after a msg is put on queue, wait 15 seconds before processing)... it is a real-time transaction ...

If you are wondering why we all look so confused, the above is why.

Let me see if I understand the requirement: a message arrives AND needs to be processed immediately (real time); processing of the input message creates a new message that is used to update a back-end database. OK, so far?

If the output message is the one that needs to be delayed, the application that consumes it can MQGET it from the queue, look at the create date/time, and wait (using o/s facilities possibly) until the 15 seconds pass since its creation, then do the back-end work.

If I didn't understand the requirement, try again.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
kevinf2349
PostPosted: Tue Mar 24, 2009 6:02 pm    Post subject: Reply with quote

Grand Master

Joined: 28 Feb 2003
Posts: 1311
Location: USA

Write an application to consume messages and only send on the ones you want it to to the back end 3rd party process. Either use a database to store the messages (thorw away duplicate keys) or use whatever criteria you use to decide it is a wasted message to throw away the worthless ones.

How are you going to determine which are the 'chosen ones?' amongst your messages and which can be discarded?

If you simply want to delay all of them then write an application with a wait coded into it....not a MQ GET WAIT... and loop around it, but this could lead to serious backlogs.

If we are voting on this as a MQ requirement in the futre ....I vote Nay!
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Mar 24, 2009 9:59 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

queuetip wrote:
we have determined we can eliminate creating a significant amount of "duplicate" data messages if we simply wait a few seconds due to the nature of the activity.


I'll ask again if these people are serious.

Waiting "a few" seconds is a ghastly way of determining this. You've clearly set "a few" at 15 but on what is that based? If the auditors come for you, what will be your defence?

If you're worried about significant numbers of "duplicate" updates to the backend system, then put a simple filtering mechanism either into the system or as a gateway in front of it. Not only is this defensible in audit terms, but will save you weeks of tuning as you struggle to determine the optimum "a few" setting.

And I have no comment on why Oracle AO has been extended with this option. Expect to point out on a previous site, with a lot of Oracle databases, the comment was "why Oracle AO?" when changing the middleware was mooted!
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
rxm8778
PostPosted: Wed Mar 25, 2009 4:24 pm    Post subject: Reply with quote

Apprentice

Joined: 15 Sep 2005
Posts: 37

My two cents on this...

I don't think delaying the processing of a message should ever be the responsibilty of MQ.

Here is an analogy...

you can't go to the post office to mail a package to a friend and ask the post office to hold the package for 10 days before sending it. If you want to truly control when the person get the package, it is up to the package sender not to go to the post office until he is ready to send the package, or it is up to the package recepient to pick up the package when he is ready, or pick it up and not open it until he is ready. Those are the three options available.

With that said you should look into delaying the message producer and not produce the message until ready....keep the message in the queue and not getting it until ready or pick up the message but delaying processing the message until ready.

That would be my approach to your requirement....

p.s you may argue that in the post office analogy, you can select overnight vs ground transportation. But remember these options were all designed to optimized the package delivery (based on how much you are willing to pay) they were not created to specifically introduce a delay betwen package posting and delivery.


Last edited by rxm8778 on Fri Mar 27, 2009 10:18 am; edited 1 time in total
Back to top
View user's profile Send private message AIM Address
Michael Dag
PostPosted: Thu Mar 26, 2009 1:30 am    Post subject: Reply with quote

Jedi Knight

Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)

actually ... I have seen this sort of requirement on MQ in the past,
it's also known as the "do not open until christmas" requirement.

basically it allows for a "pattern" to prefeed information into an organisation/unit, but all participants need to have access at the same
time, this can not be done in a normal way due to volume/processing without prefeeding (i.e. someone will get the information first or before others and will be at an advantage)

so given the above pattern I can see a "need", but no the requirement was never fulfilled.
_________________
Michael



MQSystems Facebook page
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
Vitor
PostPosted: Thu Mar 26, 2009 1:35 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

Michael Dag wrote:
actually ... I have seen this sort of requirement on MQ in the past,
it's also known as the "do not open until christmas" requirement.


Even with this, wouldn't you be better off processing the information immediately (i.e. not delaying the inbound message), warehousing the data someplace until it was complete / it was Christmas and then sending it (pub/sub??) to all interested parties? You could secure the warehouse to stop recepients "shaking the box" and getting a clue.

What an odd requirement. The ingenuity and imagination of end users is a source of constant amazement to me.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
rtsujimoto
PostPosted: Thu Mar 26, 2009 9:55 am    Post subject: Reply with quote

Centurion

Joined: 16 Jun 2004
Posts: 119
Location: Lake Success, NY

I don't want to judge whether or not your biz reqm'ts make sense or not. Sometimes the ultimate decision is truly out of your hands and it's your job to meet that reqm't. That aside, I find this sort of thing challenging. How about this:
1. Use 1 pgm to monitor the queue
2. Said pgm builds an internal table, where entry is: msgid, delay, delta
3. Table entries order by delta
4. Said pgm scans internal table, finds candidate, gets msg and puts to a
triggered queue, updates table
5. Said pgm also scans inbound queue periodically, updating table as
needed
Issues such as recoverability, speed, etc. need to be considered, but this is a start.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Fri Mar 27, 2009 9:44 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

A weird requirement, so here's a weird solution....

The consuming app is constantly checking the Oldest Message Age of the queue. Anytime it sees this value > 15 seconds, it gets one message, processes it, and then goes back to check the Oldest Message Age of the queue in a loop.

This design does not preclude multiple instances of the consuming app all running concurrently.


_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Mar 27, 2009 10:06 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

And if every single message is delayed by 15 seconds, then messages that would have conflicted 15 seconds ago because they arrived concurrently will... still conflict because they still arrived concurrently. Just 15 seconds later.

Best bet is to use WMB, and use the Timeout nodes to reprocess individual messages.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Fri Mar 27, 2009 10:10 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

Jeff, I don't understand your comment.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » IBM MQ API Support » Can You Intentionally Delay a Message's Availabilty?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.