| Author |
Message
|
| sandkul.2006 |
 Posted: Thu Jul 28, 2016 4:49 pm Post subject: Parsing DFDL message at MQInput Node using BLOB |
 |
|
Novice
Joined: 20 Mar 2015
Posts: 10
|
Hello Everyone,
As per our requirement we have only one message flow starting with MQInputNode.
We are getting COBOL Copybook as well as SOAP/XML message in same queue. We need to parse both at input node itself. Able to parse SOAP/XML using Input Message Parsing as BLOB.
When we receive DFDL message it gives below Error and failing at MQInput node itself.
Please assist me in resolving this issue. Thanks for your help in advance!
Exception:
Text:CHARACTER:Caught exception and rethrowing
Insert
Type:INTEGER:14
Text:CHARACTER:com.metlife.pc.messaging.services.dig.syncservice.DIG_MainFlow.ToBlob
ParserException
File:CHARACTER:/build/slot1/S800_P/src/DataFlowEngine/ImbRootParser.cpp
Line:INTEGER:815
Function:CHARACTER:ImbRootParser::parseNextItem
Type:CHARACTER:ComIbmMQInputNode
Name:CHARACTER:com/metlife/pc/messaging/services/dig/syncservice/DIG_MainFlow#FCMComposite_1_1
Label:CHARACTER:com.metlife.pc.messaging.services.dig.syncservice.DIG_MainFlow.MQ Input
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:2
Number:INTEGER:5902
Text:CHARACTER:Exception whilst parsing
Insert
Type:INTEGER:5
Text:CHARACTER:Root
Insert
Type:INTEGER:5
Text:CHARACTER:DFDL
Insert
Type:INTEGER:5
<text removed by moderator to improve legibility> |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Thu Jul 28, 2016 10:07 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Please edit your post and break up the last line. It makes what you are trying to say a lot friendlier.
Then this
| Quote: |
As per our requirement we have only one message flow starting with MQInputNode.
|
Tells me that the so called architect is [redacted]
This can be done but why do these people insist on making things so difficult. Probably because they don't know how to code and/or use this product in the most efficient way.
In essence, your solution is to
1) Read the message as a blob
2) look for a unique pattern of data in the message
3) process accordingly.
Because there is an identifable string in the SOAP message (on of the XML Tags for example) you could cast that as a BLOB and look for it in the original message. If it is found then you know that you have SOAP/XML. IF you don't then parse it as a DFDL type.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 29, 2016 4:16 am Post subject: Re: Parsing DFDL message at MQInput Node using BLOB |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sandkul.2006 wrote: |
| As per our requirement we have only one message flow starting with MQInputNode. |
Why is this a requirement? What's driving this?
| sandkul.2006 wrote: |
| We are getting COBOL Copybook as well as SOAP/XML message in same queue. We need to parse both at input node itself. |
Why? Why in the input node? Why not further downstream in the flow?
| sandkul.2006 wrote: |
| Able to parse SOAP/XML using Input Message Parsing as BLOB. |
No, that's tautological. The BLOB domain, by it's nature, doesn't parse anything.
| sandkul.2006 wrote: |
| Please assist me in resolving this issue. |
Tell the idiot that gave you these requirements to try thinking for a while. Once they've managed that with simple stuff (thinking about standing up, thinking about sitting down, thinking about walking) get them to think about a better design with fewer arbitrary and ridiculous requirements.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sandkul.2006 |
| Posted: Fri Jul 29, 2016 4:48 am Post subject: |
|
|
Novice
Joined: 20 Mar 2015
Posts: 10
|
Thanks for your reply guys!
First of all let me explain why this requirement came. Hope I don't confuse you:
1). Organization where I am working right now has a process where to create any resource(EG, queues etc...) they have their ETA of 6+ weeks. And lot of approval processes are involved to get it done(No Exceptions are involved), so this is not an option for us
2). We have an urgent requirement where we need to deploy these changes in 2 weeks. So we have been asked to do it with same mqinput node. And we have to do it, with no other option as of now.
Now lets come to requirement and the problem I am facing:
@smdavies99 - below is the process which I am following:
1. Whenever we receive XML message in queue we are able to receive message at and after MQInput Node with no exception (Input Message parsing is kept as BLOB), then we parse BLOB msg back to string to check for an unique data element(specific XML tag in this case) and it does work smoothly. No issues here.
2. When we get COBOL Copybook(DFDL) message (this is genrated from an existing service/flow where we are generating this DFDL message and putting into the problem queue) at input node, it's failing at MQInput node itself(remember Input Message parsing is still BLOB).
Strange thing is even if I have added break points at Failure or Catch terminal connections, it's not propagating to any of the terminal connection when tried to debug. This is the reason where we are not able to check MQRFH2 header or any other unique data element present in the input data. When I change Input Message parsing then obvious case is DFDL message is getting parsed successfully but not XML.
I agree this is crazy/stupid requirement and one shouldn't recommend this when there are other simple techniques available, but for us we are left with no options but to use this option itself.
Hope this clarifies. Any help is appreciated! Thank you again guys! |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 29, 2016 4:57 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sandkul.2006 wrote: |
| Organization where I am working right now has a process where to create any resource(EG, queues etc...) they have their ETA of 6+ weeks. And lot of approval processes are involved to get it done |
That's insane. There's nothing you can tell me about convoluted approval processes and I can get a new EG and some underlying MQ topology Dev -> Prod in 3 weeks with supporting documentation.
| sandkul.2006 wrote: |
| 2. When we get COBOL Copybook(DFDL) message (this is genrated from an existing service/flow where we are generating this DFDL message and putting into the problem queue) at input node, it's failing at MQInput node itself(remember Input Message parsing is still BLOB). |
What parse timing are you using? What character set is the COBOL in? Is there any chance it's unconverted EBCDIC from a mainframe?
| sandkul.2006 wrote: |
| Strange thing is even if I have added break points at Failure or Catch terminal connections, it's not propagating to any of the terminal connection when tried to debug. |
The debugger claims another victim. 
Run a user trace and see what's happening.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Jul 29, 2016 4:58 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I told you how to do this in my first post.
Don't parse anything in the MQInput Node. Set it up to output BLOB.
Then do the message type selection in a compute node. Then you can do the parsing there as well.
go on, give it a try. You never know, it might work.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| sandkul.2006 |
| Posted: Fri Jul 29, 2016 6:25 am Post subject: |
|
|
Novice
Joined: 20 Mar 2015
Posts: 10
|
Thanks smdavies99! I will try as per your suggestion.
@ Vitor - COBOL copybook message has nothing wrong inside and we have verified it multiple times and don't see any issue there. Character set, encoding is correct as well. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 29, 2016 6:39 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sandkul.2006 wrote: |
| @ Vitor - COBOL copybook message has nothing wrong inside and we have verified it multiple times and don't see any issue there. Character set, encoding is correct as well. |
So if viewed on the queue it's readable data and the RFH2 header you mentioned correctly describes an ASCII CCSID not an EBCDIC one?
One highly plausible theory down in flames then.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sandkul.2006 |
| Posted: Fri Jul 29, 2016 6:50 am Post subject: |
|
|
Novice
Joined: 20 Mar 2015
Posts: 10
|
Thats right Vitor! It's readable data. This is what we are using.
Please let me know if this is not correct or missed anything else here.
Encoding : 273 CodedCharSetId : 819 Format : 'MQSTR '
I even copied data from queue and compared against copybook using
RFHUtil tool and alignment looks perfect along with exact matching data size. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 29, 2016 6:58 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sandkul.2006 wrote: |
Thats right Vitor! It's readable data. This is what we are using.
Please let me know if this is not correct or missed anything else here.
Encoding : 273 CodedCharSetId : 819 Format : 'MQSTR '
I even copied data from queue and compared against copybook using
RFHUtil tool and alignment looks perfect along with exact matching data size. |
One theory down then.
What does the user trace say is happening?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
