| Author |
Message
|
| muthu121521 |
 Posted: Fri Jan 10, 2020 6:42 am Post subject: DFDL Discriminator to Select choice elements for each Occure |
 |
|
Apprentice
Joined: 31 Aug 2015
Posts: 36
|
Hi Everyone,
My DFDL looks like below structure
| Code: |
TABLE ENTRY 1..50
TABLETYPE String
TABLETYPEDESC String
TABLEDATEOPEN String
CHOICE
TABLE1INFORMATION1
FIELD1
FIELD2
FIELD2
TABLE2INFORMATION
FIELD1
FIELD2
TABLE3INFORMATION
FIELD1
TABLE4INFORMATION
|
In My above DFDL i need to select the Choice based on the value of the TABLETYPE , but the issue i am facing is when using discriminator at the choice level, I am always getting only first occurrence of the TABLE ENTRY (Root level) Table type value, I am not sure how to get TABLETYPE Value of the current item within the array of TABLE ENTRY |
|
| Back to top |
|
 |
| timber |
| Posted: Sat Jan 11, 2020 1:16 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1290
|
I can definitely help with your question, but I don't have enough information yet. Please can you do the following:
- Edit your question and put the structure of the message into [ code ] tags. This will preserve the indentation and allow me to see the the logical structure of your message
- Supply the relevant part of your DFDL xsd. Again, please be sure to use [ code ] tags else it will be unreadable. |
|
| Back to top |
|
|
| muthu121521 |
| Posted: Sun Jan 12, 2020 9:01 pm Post subject: |
|
|
Apprentice
Joined: 31 Aug 2015
Posts: 36
|
DFDL looks like below structure
| Code: |
TABLE ENTRY 1..50
TABLETYPE String
TABLETYPEDESC String
TABLEDATEOPEN String
CHOICE
TABLE1INFORMATION1
FIELD1
FIELD2
FIELD2
TABLE2INFORMATION
FIELD1
FIELD2
TABLE3INFORMATION
FIELD1
TABLE4INFORMATION |
DFDL XSD are as follows
| Code: |
<xsd:element dfdl:lengthKind="implicit" dfdl:occursCountKind="implicit" maxOccurs="50" name="TABLE_ENTRY">
<xsd:complexType>
<xsd:sequence>
<xsd:element default=" " dfdl:length="3" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="TABLETYPE" nillable="true">
<xsd:annotation>
<xsd:appinfo source="http://www.wsadie.com/appinfo">
<initialValue kind="SPACE"/>
</xsd:appinfo>
<xsd:documentation>PIC X(3) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="3"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element default=" " dfdl:length="20" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="TABLETYPEDESC" nillable="true">
<xsd:annotation>
<xsd:appinfo source="http://www.wsadie.com/appinfo">
<initialValue kind="SPACE"/>
</xsd:appinfo>
<xsd:documentation>PIC X(20) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="20"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element default="0" dfdl:decimalSigned="no" dfdl:length="8" dfdl:nilKind="literalCharacter" dfdl:nilValue="%#r00; " dfdl:textNumberPattern="00000000+" name="TABLEDATEOPEN" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC 9(8) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PIC9-Display-Zoned__int">
<xsd:minInclusive value="0"/>
<xsd:maxInclusive value="99999999"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:choice dfdl:choiceLength="387" dfdl:choiceLengthKind="explicit">
<xsd:element dfdl:lengthKind="implicit" name="TABLE1INFORMATION">
<xsd:complexType>
<xsd:sequence>
<xsd:element default=" " dfdl:length="16" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD1" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC X(16) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="16"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element default=" " dfdl:length="15" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD2" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC X(15) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="15"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element default=" " dfdl:length="15" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD3" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC X(15) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="15"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element default=" " dfdl:length="15" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD4" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC X(15) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="15"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element dfdl:lengthKind="implicit" name="TABLE2INFORMATION">
<xsd:complexType>
<xsd:sequence>
<xsd:element default=" " dfdl:length="5" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD1" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC X(5) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="5"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element default="0" dfdl:decimalSigned="no" dfdl:length="8" dfdl:nilKind="literalCharacter" dfdl:nilValue="%#r00; " dfdl:textNumberPattern="00000000+" name="FIELD2" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC 9(8) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PIC9-Display-Zoned__int">
<xsd:minInclusive value="0"/>
<xsd:maxInclusive value="99999999"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element dfdl:lengthKind="implicit" name="TABLE3INFORMATION">
<xsd:complexType>
<xsd:sequence>
<xsd:element default=" " dfdl:length="5" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD1" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC X(5) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="5"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element default=" " dfdl:length="15" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD2" nillable="true">
<xsd:annotation>
<xsd:documentation>PIC X(15) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="15"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element default=" " dfdl:length="387" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="TABLE4INFORMATION" nillable="true">
<xsd:annotation>
<xsd:appinfo source="http://www.wsadie.com/appinfo">
<initialValue kind="SPACE"/>
</xsd:appinfo>
<xsd:documentation>PIC X(387) display</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="dfdlCobolFmt:PICX__string">
<xsd:maxLength value="387"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:choice>
</xsd:sequence>
</xsd:complexType>
|
But this is the Structure , i am actually not able to get the values of the Tabletype in each iteration of the array , i was able to get only index of the any only one iteration , i was not able to loop through the array in discriminator |
|
| Back to top |
|
|
| timber |
| Posted: Mon Jan 13, 2020 2:01 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1290
|
Thanks - that's much clearer now.
The 'TABLETYPE' field is acting exactly like a tag (or 'initiator' in DFDL language). I suggest that you model this as follows:
| Code: |
message
...
element name="TABLE_ENTRY" maxOccurs=50
choice initiatedContent="true"
sequence initiator="ABC"
group ref="commonFields" (TABLEDESC AND TABLEOPEN)
<other fields in 1st choice branch>
sequence initiator="DEF"
group ref="commonFields" (TABLEDESC AND TABLEOPEN)
<other fields in first 2nd choice branch>
sequence initiator="GHI"
group ref="commonFields" (TABLEDESC AND TABLEOPEN)
<other fields in 3rd choice branch>
...etc
|
This will also be quite a lot faster to parse because the parser will extract the TABLETYPE field once and resolve the choice immediately (that's what the initiatedContent="true" does). |
|
| Back to top |
|
|
| muthu121521 |
| Posted: Mon Jan 13, 2020 7:45 am Post subject: |
|
|
Apprentice
Joined: 31 Aug 2015
Posts: 36
|
Hi Timber,
Thanks for Your Reply , due to security issue i was not allowed to put the actual DFDL schema, but the original DFDL schema (Copybook) is very length so i have shortened it to make only the issue highlight but the actual original DFDL looks like below format , each choice has more than 10 fields
| Code: |
TABLE ENTRY 1..50
TABLENO String
TABLETYPE String
TABLETYPEDESC String
TABLEDATEOPEN String
ELEMENT1 STRING
.....
.....
ELEMENT24 STRING
CHOICE
TABLE1INFORMATION
ELEMENT1 STRING
ELEMENT2 STRING
...
ELEMENT13 STRING
FILLER
TABLE2INFORMATION
ELEMENT1 STRING
ELEMENT2 STRING
...
ELEMENT8 STRING
FILLER
TABLE3INFORMATION
ELEMENT1 STRING
ELEMENT2 STRING
...
ELEMENT28 STRING
FILLER
TABLE4INFORMATION
ELEMENT1 STRING
ELEMENT2 STRING
...
ELEMENT3 STRING
FILLER
TABLE5INFORMATION
ELEMENT1 STRING
ELEMENT2 STRING
...
ELEMENT15 STRING
FILLER
TABLE6INFORMATION
ELEMENT1 STRING
ELEMENT2 STRING
...
ELEMENT45 STRING
FILLER
TABLE7INFORMATION
|
Each Choice all are of equal length of 387 bytes. Main issue i have faced when i am using discriminator or choice dispatch key , i was not able to determine the Table Type values in the array , when i am trying to use XPath Expression Builder , when i tried to drag Tabletype field from Tabletype entry , its forcing me to give an index, table type value can be different for each occurence of the TABLE ENTRY array, how can i overcome that , not sure of retrieveing the respective table type value? |
|
| Back to top |
|
|
| timber |
| Posted: Mon Jan 13, 2020 8:31 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1290
|
OK - thanks for making that clear. I will adjust my advice and make it clearer. If you follow this approach, you will not need to solve your XPath problems because you will not need to write any discriminator expressions. You will need to adjust the structure of the DFDL schema a little, though.
1. If you are using TABLETYPE as a initiator then you need to remove the TABLETYPE element from the DFDL schema. Otherwise DFDL will consume the 3 bytes of TABLETYPE to use as the initiator, then it will try to parse the same 3 bytes again as the TABLETYPE element (but it will actually be consuming the next field instead). Sorry for not explaining that before.
2. If you are using TABLETYPE as an initiator then you do not need any discriminators. The initiator identifies the choice branch automatically.
3. If you are using TABLETYPE as an initiator then you must define the global group 'commonFields' and add a reference to commonFields into every branch of the choice group.
4. If you are using TABLETYPE as an initiator then you must set the initiator on every branch of the choice. The initiator value is whatever value TABLETYPE would have for that choice branch.
So, let me adjust my suggested structure for you:
| Code: |
-- This global group contains all of the fields _after_ TABLETYPE and _before_ the CHOICE group.
group name="commonFields"
element name="TABLEDESC"
element name="TABLEOPEN"
ELEMENT1 STRING
.....
.....
ELEMENT24 STRING
element name="TABLE_ENTRY" maxOccurs=50
choice initiatedContent="true"
sequence initiator="ABC"
group ref="commonFields" (TABLEDESC AND TABLEOPEN)
<other fields in 1st choice branch>
sequence initiator="DEF"
group ref="commonFields" (TABLEDESC AND TABLEOPEN)
<other fields in first 2nd choice branch>
sequence initiator="GHI"
group ref="commonFields" (TABLEDESC AND TABLEOPEN)
<other fields in 3rd choice branch>
...etc
|
If you don't want to use this technique, then you will need to find a way write the discriminator expression. I think the position() function is probably the answer. But I would try using the initiators first - it will use a lot less CPU. |
|
| Back to top |
|
|
| muthu121521 |
| Posted: Mon Jan 13, 2020 9:38 am Post subject: |
|
|
Apprentice
Joined: 31 Aug 2015
Posts: 36
|
We have always used the copybook and converted into DFDL Schema in IIB Creating new message model utility , Can you give me sample dfdl group reference declaration ?
Can we provide multiple values to check if either ABC or BED Then TABLE1Information ? |
|
| Back to top |
|
|
| timber |
| Posted: Tue Jan 14, 2020 1:05 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1290
|
| Quote: |
| Can you give me sample dfdl group reference declaration? |
You actually need a global group declaration, and then multiple group references (one in each branch of the choice). A DFDL group/group ref is no different from an XSD group/group ref. Just declare the group and reference it in the usual way:
| Code: |
<xsd:group name="commonFields">
<xsd:sequence>
COPY AND PASTE TABLEDESC, TABLEOPEN ETC HERE
</xsd:sequence>
</xsd:group> |
and the group reference:
| Code: |
| <xsd:group ref="myPrefix:commonFields" /> |
...and (obviously) change myPrefix to whatever the correct prefix is. You can just copy the common fields, with all of their dfdl attributes and annotations, and paste them into the global group.
| Quote: |
| Can we provide multiple values to check if either ABC or BED |
So the first choice branch can have two different values for TABLETYPE? No problem - DFDL allows the initiator to be specified as a space-separated list of alternative values. Just specify the "ABC BED" as the initiator value. |
|
| Back to top |
|
|
| muthu121521 |
| Posted: Tue Jan 14, 2020 7:48 am Post subject: |
|
|
Apprentice
Joined: 31 Aug 2015
Posts: 36
|
Hi Timber,
Thanks for that suggestion, it really worked !! thanks very much for your help on this.
But i am facing couple issues here
1.Last choice which is TABLE4INFORMATION unlike the other choices , it doesn't have any subset of elements , i want this choice to get populated when every other choices failed, what value i can provide here in the initiator?
2.But even if i provide some random value in the initiator , I am getting the below error
Regardless of the value i set i keep getting below errors, I have even tried the quick fix for
| Code: |
| CTDV1561E : When 'initiatedContent' is 'yes' for the sequence or choice, DFDL property 'emptyValueDelimiterPolicy' must not be 'none' or 'terminator'. Element: #xscd |
| Code: |
| CTDV1559E : When 'lengthKind' is 'explicit' or 'implicit', DFDL property 'emptyValueDelimiterPolicy' must not be 'initiator' or 'both' |
3. I was also trying Below discriminator expression for one Choice example
| Code: |
{fn:contains( 'ABC:BDE',/TABLE ENTRY[fn:position()]/TABLETYPE}
|
To determine the choice based on the Table type values in each Table Entry Occurence , i was not able to evaluate the expression, just want to whether it is fn:position() or dfdl:position , because i have tried them both and both of them are not working at all , Just wanted to get the correct rexpression actually |
|
| Back to top |
|
|
| timber |
| Posted: Tue Jan 14, 2020 2:33 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1290
|
1. Ah. That means that we cannot set initiatedContent="true" because that optimisation relies on the parser knowing all of the initiators in advance.
2. The error message is telling ththe current value of emptyValueDelilmiterPolicy is not compatible with initiatedContent="true". However...we will need to set initiatedContent="false" because of 1. So this error should go away.
3. If you want to return to the original DFDL schema structure, and make the discriminators work then you will need the DFDL function dfdl:occursIndex().
See the DFDL specification https://www.ogf.org/documents/GFD.207.pdf for the details of other functions (section 23)
Since you have almost got the initiators working correctly, you might as well try this first:
- Set initiatedContent="false" on the choice group. This should fix the error in your question 2.
- Do not define any initiator for the final choice branch. In fact, you should probably explicitly set it to the empty string since your parsing strategy relies on it being empty.
- In the final choice branch, add the element declaration for TABLETYPE into the sequence group as the first member (immediately before the commonFieldsgroup ref). |
|
| Back to top |
|
|
| muthu121521 |
| Posted: Wed Jan 15, 2020 6:34 am Post subject: |
|
|
Apprentice
Joined: 31 Aug 2015
Posts: 36
|
Thanks for that suggestion.
| Quote: |
1. Ah. That means that we cannot set initiatedContent="true" because that optimisation relies on the parser knowing all of the initiators in advance.
2. The error message is telling ththe current value of emptyValueDelilmiterPolicy is not compatible with initiatedContent="true". However...we will need to set initiatedContent="false" because of 1. So this error should go away. |
If we set initiatedContent="false" or initiatedContent="No" at the root choice level wont it impact for rest of the choices option?
Regarding with the Point 3
| Quote: |
If you want to return to the original DFDL schema structure, and make the discriminators work then you will need the DFDL function dfdl:occursIndex().
See the DFDL specification https://www.ogf.org/documents/GFD.207.pdf for the details of other functions (section 23) |
We are using IIB Version 10 , in the DFDL Section I can see that occursIndex feature coming under unsupported feature as in the below link
https://www.ibm.com/support/knowledgecenter/SSMKHH_10.0.0/com.ibm.etools.mft.doc/df00150_.html
Because I have even tried as you suggested in the discriminator section and it throws me error when i try to use the below expression
| Code: |
| {fn:contains( 'ABC:BDE',/TABLE ENTRY[dfdl:occursIndex()]/TABLETYPE} |
Any Correct Expression suggestion would be of hugely helpful |
|
| Back to top |
|
|
| timber |
| Posted: Wed Jan 15, 2020 9:44 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1290
|
| Quote: |
| We are using IIB Version 10 , in the DFDL Section I can see that occursIndex feature coming under unsupported feature |
Oops! Well spotted - I forgot that occursIndex was not supported.
| Quote: |
| If we set initiatedContent="false" or initiatedContent="No" at the root choice level wont it impact for rest of the choices option? |
Yes, but that is not as serious as you think. The property 'initiatedContent' switches on an optimisation in the DFDL parser, and makes it assume that every branch of the choice has a fixed-length initiator, [i]and[/] all initiators must have exactly the same length. This allows the parser to read the initiator bytes once, and quickly compare the resulting byte values against each choice branch.
If you set it to 'No' then the parser must read the initiator once for each branch. If the TABLETYPE bytes for a branch do not match the initiator then DFDL will backtrack to the start of the choice before starting to parse the next branch. Less efficient, but it still works. And you need that behavour in your scenario because final branch does not have an initiator. |
|
| Back to top |
|
|
|
|
