| Author |
Message
|
| Vitor |
 Posted: Fri Apr 17, 2009 6:03 am Post subject: |
 |
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| goffinf wrote: |
I guess I can do a non-destructive read using correlation id before the put and see if I get anything.
|
Not if you're putting or publishing to a remote queue. And CorrelId shouldn't contain business data either.
Especially in pub/sub where IIRC it's used by the system.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
 |
| goffinf |
| Posted: Fri Apr 17, 2009 6:13 am Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| Quote: |
| Not if you're putting or publishing to a remote queue |
Well obviously the Get will be local (I can do that no problem).
| Quote: |
| And CorrelId shouldn't contain business data either |
Why not ! What if I was doing insurance quotes and I wanted to use [say] the policy number as a way of correlating messages. Why wouldn't I use that (obviously formatted to be a valid value for MQ) ? Whatever the value used for correlation purposes, its just a token right ?
| Quote: |
| Especially in pub/sub where IIRC it's used by the system. |
The check for duplicates happens BEFORE publication, and publication should ONLY occur for NON duplicate.
Fraser. |
|
| Back to top |
|
|
| exerk |
| Posted: Fri Apr 17, 2009 6:22 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| goffinf wrote: |
| Why not ! What if I was doing insurance quotes and I wanted to use [say] the policy number as a way of correlating messages. Why wouldn't I use that (obviously formatted to be a valid value for MQ) ? Whatever the value used for correlation purposes, its just a token right ? |
Oh dear...I have just definitely had my vu deja'd!
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| goffinf |
| Posted: Fri Apr 17, 2009 7:09 am Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| Quote: |
| Oh dear...I have just definitely had my vu deja'd! |
You're talking in riddles. Can you give me a rational explanation why I shouldn't use CorrelID as an accessor value for a message, and why I shouldn't put a value that I can easily derive from business data in there.
Sure I could let MQ create me the value and pass it back to the requestor and say 'hey .. any time you talk to me in the future, pass me this value', but thats not the situation I'm in and in any event I don't see any real difference between MQ creating the value and doing it myself (I can apply as much uniqueness to it as I need).
Fraser. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Apr 17, 2009 7:12 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| goffinf wrote: |
| You're talking in riddles. Can you give me a rational explanation why I shouldn't use CorrelID as an accessor value for a message, and why I shouldn't put a value that I can easily derive from business data in there. |
A CorrelationID is not a string. A CorrleationID is not a number. A correlation ID is always 24 bytes long.
Your business data is almost never going to meet that criteria. Putting business data in CorrelId and MsgId is a long established WORST practice that has shown over and over and over again to cause unexpected problems in the middle of the night during a busy production cutover window that are unfixable for three months until the next development cycle completes. |
|
| Back to top |
|
|
| exerk |
| Posted: Fri Apr 17, 2009 7:17 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
 Thank you mqjeff - I would not have been that erudite.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Apr 17, 2009 7:33 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
All of that said, and hopefully well received...
For the purpose of the solution at hand, using a queue that held messages with JUST the actual business data field in them, and some kind of generated/calculated hash key from the business data field in the correlId is a reasonable approach, as long as a lot of care is taken to ensure that the correlID field that is generated and is "safe". |
|
| Back to top |
|
|
| goffinf |
| Posted: Fri Apr 17, 2009 8:57 am Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| Quote: |
| All of that said, and hopefully well received... |
Of course.
| Quote: |
| A CorrelationID is not a string. A CorrleationID is not a number. A correlation ID is always 24 bytes long. |
Yes, I'm well aware of that, but thats not really a problem.
| Quote: |
| and some kind of generated/calculated hash key from the business data field in the correlId is a reasonable approach |
Indeed, that is exactly what I meant by :-
| Quote: |
| (obviously formatted to be a valid value for MQ) |
from one of my earlier responses.
We have done this a few times, that is, take a piece of BUSINESS DATA that is present and consistent in all the messages exchanged (and has been validated) and (carefully) create a 24 byte hash, and use that as the correlation id. I've never had a problem with it thus far, thats why I was surprised to see such a backlash. As I said, at the end of the day its just a 24 byte 'token' that an application can straight-forwardly reproduce from data that it has available.
So, it seems that I have at least one potential solution, albeit admittedly not ideal.
Fraser. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Apr 17, 2009 11:50 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20767
Location: LI,NY
|
| Quote: |
The check for duplicates happens BEFORE publication, and publication should ONLY occur for NON duplicate.
Fraser. |
Still not getting it. I am surprised that Jeff did not point it out either.
Pub/Sub uses the correlationId field for its own purpose. If you want to pass your own data don't use the correlationId field use a user property on the RFH2 to store it. Beware of the fact that if you have to search for the property you should be using MQ V7 as in previous versions that search on the queue is going to be very expensive.... Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Sat Apr 18, 2009 10:36 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| goffinf wrote: |
| I've never had a problem with it thus far |
This day will come.
| goffinf wrote: |
| I was surprised to see such a backlash. |
I'm afraid this question has come up a lot.
| goffinf wrote: |
| As I said, at the end of the day its just a 24 byte 'token' that an application can straight-forwardly reproduce from data that it has available. |
Currently. I will say you have a far more sophisticated technique than most.
| goffinf wrote: |
| So, it seems that I have at least one potential solution, albeit admittedly not ideal. |
It's never ideal to set msgid or correlid, and is a cause of problems when you do. In the end you have to do what is best for you; we can just advise.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Apr 19, 2009 12:43 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20767
Location: LI,NY
|
| goffinf wrote: |
| We have done this a few times, that is, take a piece of BUSINESS DATA that is present and consistent in all the messages exchanged (and has been validated) and (carefully) create a 24 byte hash, and use that as the correlation id. I've never had a problem with it thus far, thats why I was surprised to see such a backlash. As I said, at the end of the day its just a 24 byte 'token' that an application can straight-forwardly reproduce from data that it has available. |
So did you ever use it in a pub/sub application? What was the result?
As I remember the pub/sub engine uses the correlationId field to distinguish between subscriptions at least in some cases. As such there is no reason for YOUR correlationId to be preserved on the message being served to the subscriber... as he is decoupled from the producer. If you require this field value to be received by the subscriber put it as a user defined property of the JMS message.
Caveat about searches on said property in version lower than V7. You might have to write a little app to remove this value and put it to the correlid field before your application consumes the message. Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sun Apr 19, 2009 6:34 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I don't think that goffinf is actually talking about setting the correlid on the actual business message that is being published.
Just in (mis) using CorrelId to use a queue as a lookup table, instead of being forced to use an additional persistance mechanism. That is, it sounds like he's going to create a new, small, message that goes on an application-private queue, and just use that to find out if the current message SHOULD be published.
It seems like goffinf has the necessary skills and understanding to get away with this, sufficiently far beyond what the usual "why doesn't my correlationID get converted on the mainframe" kind of poster. So it's less likely to cause him (sorry, I'm pretty sure Frasier is a male name) to get woken up in the middle of the night because "MQ Lost my message".
But it doesn't remove the onus on the subscriber applications from having to deal with duplicated messages. Something is still going to slip through, somewhere sometime. So, sure, put in some additional validation and control at the middleware layer, but don't let the subscriber apps off the hook for their own necessary validation and data safety requirements. |
|
| Back to top |
|
|
|
|
