| Author |
Message
|
| kimbert |
 Posted: Fri Jul 13, 2012 2:27 am Post subject: |
 |
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5543
Location: Southampton
|
Looks as if you have got quite a long way down the road on your own. Not bad for somebody who's new to MRM/TDS.
| Quote: |
How do I go about splitting these pairs of lines? Just getting each pair of lines into a some kind of a complex element would do. From there, they can be further parsed.
|
I would define a group like this
| Code: |
sequence DataElementSeparation="Tagged Delimited" delimiter="<CR><LF>SI"
element name="Leg" minOccurs="0" maxOccurs="unbounded"
complexType DataElementSeparation="Tagged Delimited" delimiter="<CR><LF>"
element name="firstLine" Tag="-"
complexType DataElementSeparation="All Elements Delimited" delimiter="."
element name="field1"
element name="field2"
element name="secondLine" Tag="."
complexType DataElementSeparation="All Elements Delimited" delimiter="."
element name="field1"
element name="field2" |
If you haven't already discovered it, here's a tip. When your model doesn't work as expected, a debug-level user trace can be very helpful. Especially when combined with a Trace node in your flow that has the pattern set to ${Root}. |
|
| Back to top |
|
 |
| whydieanut |
| Posted: Fri Jul 13, 2012 3:40 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
I tried using the <CR><LF>SI delimiter, but though I could get all the stuff for the Legs into a structure, I was unable to process them further; must have surely done something wrong.
Then remembered the sticky post on this forum about TDS parsers; imported it and referred it.
My head is spinning, but this is what I have got so far...
Am pretty sure I am not gonna remember a thing of how I got this on Monday... 
Sample Message:
| Code: |
QU JPSZZAI
.BOMRCAI 121049
LDM
AI126/11.VTALN.F4C35Y303.2/13
-FRA.0/0/0/0.0.T78.4/78
.PAX/0/0/0.PAD/0/0/0
-DEL.83/59/4/0.0.T7930.1/2037.3/3279.4/2414.5/200
.PAX/0/8/138.PAD/0/0/5.RNG/32R
-HYD.34/24/2/3.0.T3187.3/3187
.PAX/0/0/60.PAD/0/0/0
SI FRA B/NIL.C/78.M/NIL.E/NIL
DEL B/6853.C/1077.M/NIL.E/NIL
HYD B/3187.C/NIL.M/NIL.E/NIL
STAB TRIM SETTINGS USED FLAP 5
STAB TRIM SETTINGS FOR FLAP 15 3.1
BAGGAGE-8246
CARGO-562
MAIL-0
NOTOC-YES-RNG LDD IN POS 32R
;
|
Debug Output:
| Code: |
MRM
H1
H1.1:CHARACTER:QU
H1.2:CHARACTER:JPSZZAI
H2
H2.1:CHARACTER:.BOMRCAI
H2.2:CHARACTER:121049
MSG_TYPE:CHARACTER:LDM
H4
H4.1
FLTNO:CHARACTER:AI126
DAY:CHARACTER:11
TAILNO:CHARACTER:VTALN
SEAT_CAPACITY
CLASS1
CLASS:CHARACTER:F
SEATS:CHARACTER:4
CLASS2
CLASS:CHARACTER:C
SEATS:CHARACTER:35
CLASS3
CLASS:CHARACTER:Y
SEATS:CHARACTER:303
H4.3
CD:CHARACTER:2
CC:CHARACTER:13
LEG
LEG_1
DST:CHARACTER:FRA
SEATS:CHARACTER:0/0/0/0
ZERO:CHARACTER:0
T:CHARACTER:T78
HOLD:CHARACTER:4/78
LEG_2
PAX:CHARACTER:PAX/0/0/0
PAD:CHARACTER:PAD/0/0/0
LEG
LEG_1
DST:CHARACTER:DEL
SEATS:CHARACTER:83/59/4/0
ZERO:CHARACTER:0
T:CHARACTER:T7930
HOLD:CHARACTER:1/2037
HOLD:CHARACTER:3/3279
HOLD:CHARACTER:4/2414
HOLD:CHARACTER:5/200
LEG_2
PAX:CHARACTER:PAX/0/8/138
PAD:CHARACTER:PAD/0/0/5.RNG/32R
LEG
LEG_1
DST:CHARACTER:HYD
SEATS:CHARACTER:34/24/2/3
ZERO:CHARACTER:0
T:CHARACTER:T3187
HOLD:CHARACTER:3/3187
LEG_2
PAX:CHARACTER:PAX/0/0/60
PAD:CHARACTER:PAD/0/0/0
SI:CHARACTER:SI FRA B/NIL.C/78.M/NIL.E/NIL
END:CHARACTER:DEL B/6853.C/1077.M/NIL.E/NIL
END:CHARACTER:HYD B/3187.C/NIL.M/NIL.E/NIL
END:CHARACTER: STAB TRIM SETTINGS USED FLAP 5
END:CHARACTER: STAB TRIM SETTINGS FOR FLAP 15 3.1
END:CHARACTER: BAGGAGE-8246
END:CHARACTER: CARGO-562
END:CHARACTER: MAIL-0
END:CHARACTER: NOTOC-YES-RNG LDD IN POS 32R
END:CHARACTER:
END:CHARACTER:;
|
|
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Jul 13, 2012 4:07 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5543
Location: Southampton
|
Great progress. Not sure whether you're happy with the result so far. You didn't actually mention any more problems.
One thing I almost mentioned before, but decided not to confuse you with details. There are several fields like this
| Code: |
| FRA B/NIL.C/78.M/NIL.E/NIL |
where the subfields have not been separated out.
The way to deal with this is to insert a local sequence group into the complex type as a *child* of the main sequence group. Set its delimiter to '/' and add the various subfields as child elements. That way, you will get the subfields separated out in the message tree. It's a powerful technique - you can do many useful things in TDS by adding a local sequence that overrides the properties of its parent. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Fri Jul 13, 2012 4:29 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
Thanks kimbert,
Will have a look at it too.
The whole message modelling business on WMB is sooooo vast...
Too many things to know and understand.
For the time being seems, what I have is enough for this particular message type. Anyways has been a good learning experience.
Will look at the next message type on Monday...
Also want to look at when to use Groups, how to use the choice operator, etc... |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Jul 13, 2012 4:44 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5543
Location: Southampton
|
| Quote: |
| The whole message modelling business on WMB is sooooo vast... |
Yep. It's positively scary. Until you move to v8 and use DFDL. Then it all seems eerily easy  |
|
| Back to top |
|
|
| shanson |
| Posted: Fri Jul 13, 2012 5:25 am Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
| I have a DFDL schema for MATIP Type B messages. It's a first pass and treats all the Address section and Text section lines as string data, but it's a start. It handles the optional Diversion and ShortAddress sub-sections of the Address section. If anyone is interested append your email and I'll send it. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Mon Jul 16, 2012 3:24 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
I am back!
Trying to parse another type of message now.
| Code: |
COMAI
AI126/11JUL ORD/0 CC/NAM
CWT262143 UWT238453 CAW262143 UAW238453 WTN1 KG TCI
77W/1 GTD/M10 POS/GATE BDT1300 SD1345 AD1425
ARN/VTALN CNF/F4C35Y303 CAP/F4C29Y273 AV/F4C21Y75
PADC/F0C0Y5
ZONES - 0A/000 0B/008 0C/059 0D/078 0E/061
FRA 0/0/0/0 0/0/0 0/0/0 0/0/0 0/0 0/0 0/0/0
DEL 83/59/4/0 252/5382/0 0/8/138 0/0/5 0/0 0/0 0/0/0
HYD 34/24/2/3 118/2569/0 0/0/60 0/0/0 0/0 0/0 0/0/0
TTL 117/83/6/3 370/7951/0 0/8/198 0/0/5 0/0 0/0 0/0/0 0 15670
|
This needs to be split as below:
(Only interested in the fields enclosed in brackets () )
| Code: |
COMAI
(AI126) (11JUL) (ORD)/0 CC/NAM
(CWT: 262143) (UWT: 238453) (CAW: 262143) (UAW: 238453) (WTN1 KG TCI)
(77W) 1 GTD/M10 POS/GATE (BDT: 1300) (SD: 1345) (AD: 1425)
(ARN) (VTALN) (CNF: F4,C35,Y303) (CAP: F4,C29,Y273) (AV: F4,C21,Y75)
PADC/F0C0Y5
(ZONES: 0A:000, 0B:008, 0C:059 0D:078 0E:061)
(FRA) 0/0/0/0 0/0/0 0/0/0 0/0/0 0/0 0/0 0/0/0
(DEL) 83/59/4/0 252/5382/0 0/8/138 0/0/5 0/0 0/0 0/0/0
(HYD) 34/24/2/3 118/2569/0 0/0/60 0/0/0 0/0 0/0 0/0/0
TTL (117, 83, 6, 3) 370/7951/0 (0, 8, 198) 0/0/5 0/0 0/0 (0, 0, 0) 0 15670
|
The problem is that some of the lines have varying no. of white spaces; so I am not sure what to set as the delimiter.
Can use the Data Pattern separator, but again not sure if that is the most efficient way to do it.
Any comments on using Data Patterns? |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Jul 16, 2012 3:42 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5543
Location: Southampton
|
| Quote: |
Can use the Data Pattern separator, but again not sure if that is the most efficient way to do it.
Any comments on using Data Patterns? |
Yes.
1. Don't be scared of using them when you need to. They do consume more CPU than, say, 'All Elements Delimited'. But they're a lot more efficient now (since v6.0) than they were in earlier versions.
2. When applied to a structure ( a complex element or a group ), a data pattern must match all of the structure. Why? Because the data pattern tells the TDS parser how long the structure is.
3. If a structure has Data Element Separation="Use Data Pattern" you can still parse its sub-fields as delimited fields. Just insert a sequence group with Data Element Separation="All Elements Delimited". Using this technique you can limit the use of data patterns to those parts of the model that really need it. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Jul 16, 2012 3:47 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Also, you may find that using a whitespace as a delimiter merely ends up giving you extra, empty, elements. This might not be a significant issue. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Mon Jul 16, 2012 3:50 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
Got it, thanksQ!
Will try out and report the progress...
 |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Jul 16, 2012 3:53 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5543
Location: Southampton
|
Oh - I almost forgot to mention this.
The DFDL language explicitly supports
- delimiters ( including tags and terminators ) that contain an unknown amount of white space.
- flexible matching of any style of linefeed ( using the %NL; entity )
- a set of alternative delimiters ( or tags or terminators ) for any element or group. MRM only allows one. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Mon Jul 16, 2012 4:06 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
If I were to look at this particular line:
CWT262143 UWT238453 CAW262143 UAW238453 WTN1 KG TCI
I want it to be parsed as:
CWT: 262143
UWT: 238453
CAW: 262143
UAW: 238453
WTN1
KG
TCI
I can't use the Tagged Delimited separator, can I? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Jul 16, 2012 4:23 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| whydieanut wrote: |
If I were to look at this particular line:
CWT262143 UWT238453 CAW262143 UAW238453 WTN1 KG TCI
I want it to be parsed as:
CWT: 262143
UWT: 238453
CAW: 262143
UAW: 238453
WTN1
KG
TCI
I can't use the Tagged Delimited separator, can I? |
At least for some of it, you can. As kimbert keeps saying, you can embed as many local sequences as you want in other sequences. So you can have a sequence that covers the whole section, that contains a sequence that is tagged/delimited and covers the CWT->UAW, and then the outer sequence has some fixed-length fields or just delimited fields.
But it also depends on which elements are optional or not. |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Jul 16, 2012 4:45 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5543
Location: Southampton
|
mqjeff is correct. You need a complex type with Data Element Separation="Tagged Delimited". Within that, you can add the first four ( tagged) fields and set their tags. The remaining three fields don't have a tag, but that's no problem - just add a local sequence that contains them, and set Data Element Separation="All Elements Delimited". In the message tree, all seven fields will appear under the same parent element.
It sounds simple, but the details matter. Can you have more than one space between the elements? If so, then you probably need data patterns. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Mon Jul 16, 2012 4:54 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
For the fisrt 4 elements I have done something like below.
COM_LINE3 is All elements delimited.
It has 7 child elements
The first 4 are local complex types (Tagged delimited)
The last 3 are simple string fields
| Code: |
COM_LINE3 (All elements delimited)
+-CWT
| +-{Local Complex Type} (Tagged Delimited)
| +-CWT
+-UWT
| +-{Local Complex Type} (Tagged Delimited)
| +-UWT
+-CAW
| +-{Local Complex Type} (Tagged Delimited)
| +-CAW
+-UAW
| +-{Local Complex Type} (Tagged Delimited)
| +-UAW
+-WTN1
+-KG
+-TCI
|
Looks ugly with some unnecessary nesting.
Is there a better way of doing this? |
|
| Back to top |
|
|
|
|
