| Author |
Message
|
| Elmine |
 Posted: Fri Aug 15, 2003 4:27 am Post subject: Fixed Length Tagged/Delimited Format |
 |
|
Newbie
Joined: 12 Aug 2003
Posts: 9
Location: South Africa
|
Hi
I've got a message set with this format and fixed lenght. I've declared a message with repeating compound types. I assign values to the elements of the repeating compound type in my compute node. In the debugger, I can see these values, but get the error 'Mandatory element has no value or default value assigned' when I try to put the message on a queue. According to me none of the elements should be mandatory as I only supplied a value for max occurs. Any ideas why this is happening? |
|
| Back to top |
|
 |
| Craig B |
| Posted: Fri Aug 15, 2003 6:23 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
When dealing with elements that have a Fixed length data element separation then all elements have to be considered mandatory. This is because TDS parser will be instructed to read out the next N bytes for the current field, where N is the length in the field. So in a case where you have three fields each of length 6 and you pass in a bitstream that was Value1Value2Value3, then Value1 would be allocated to Field 1, and Value2 would be allocated to Field 2 etc. If in your mind you omit Value2 from the bitstream, then this would lead to Value1 being allocated to Field1, and now the next bytes for Value2 will be allocated to Field2, and this would then leave no bytes for Field 3. In the TDS layer, if there is no data for elements at the end of the structure, and the end of the structure is distinguishable, then end elements can be omitted.
So how does this apply to your scenario? Well in this case you have a repeating stucure that reads in a message, and parses N number of repetitions. The number of repetitions is bounded by your maxOccurs value, but you dont have that many repetitions, such that your repetitions are bounded but of an unspecified number. I would imagine in your case, that the number of repetitions takes up the whole input message, and their are no fields afterwards. When the TDS parser reads in your number of repetitions then it will eventually hit the end of the bitstream. If the maxOccurs number of repetitions has not been parsed, then the TDS parser will see the other repetitions has being omitted and any subsequent fields that are in the model.
This explains the parsing side. Now on to your 'Mandatory element has no value or default value assigned' error. This will be produced by the TDS writer when a new output message is being created. It seems the TDS writer is far more strict that TDS parser. It knows that it has to write messages, that its own TDS parser can reparse. Therefore, how can it write out an unspecified number of repetitions such that the message can be reparsed, and the number of repetitions can be parsed again. In this type of case, it follows the only known value it has, and attempts to write the maxOccurs number of repetitions. Therefore if your message tree only contains 5 instances, and maxOccurs is 1000 then the TDS writer will want to write 1000 records. If these are not in your message tree then it will want to use default values to populate the remaining records. If you have no default values assigned then it will throw the error you are seeing because it sees all of them as being mandatory.
It comes down to the fact that in a Fixed length only environment you cannot have an unspecified number of repetitions. The TDS parser may look like it can handle this, but this by the virtue that it runs out of data, not that it knew how many there were. If the example were changed to the following :
| Code: |
- Element1 : Length=5, Repeats 0 to 1000 times
- Element2 : Length=5
|
Then the question would be ... if Element1 can repeat any number of times between 0 and 1000, .... how do you know where Element1's finish and Element 2 starts. It could be argued that the last five bytes could be taken first, and then Element1 parsed from the rest, so lets present a more complicated example :
| Code: |
- Element1 : Length=5, Repeats 0 to 1000 times
- Element2 : Length=5, Repeats 0 to 1000 times
|
In this model, it is impossible for the TDS parser to know where one element finishes and the next one starts. Therefore the TDS writer has to write out all occurences (up to MaxOccurs) to guarantee it can reparse it all.
I would imagine that this type of modelling is allowed in the TDS parser because Fixed length separation can be nested with other separation etc, such that the end of unbounded repetitions is known.
It would seem that there is room for improvement such that if you are dealing with the last element in a message, then it should be able to cope with unbounded repetitions since the end of the message would signal the end of the repetitions. The CWF parser supports an End of Bitstream for its length units, it seems that the TDS parser would benefit from a repeat until end of bitstream option.
_________________
Regards
Craig |
|
| Back to top |
|
|
| kirank |
| Posted: Fri Aug 15, 2003 8:53 am Post subject: |
|
|
Centurion
Joined: 10 Oct 2002
Posts: 136
Location: California
|
You can resolve this problem by assigning a Default value to all the elements in the message. Create an Element Value say 'Default'. The actual value for this Element Value can be nothing, just keep it blank.
Once you have cretaed the 'Default' Element value, for each element within your message, select Add >Value Constraint>Default and add the element value you have created. Once this changed message set is deployed you should be fine.
For Fixed length data element separation all elements are considered mandatory so you should assign a value to each element in a compute node or have a default value as explained. If the default value is blank, then you will get e.g 3 blank spaces in output if that particular element has a length of 3.
Hope this helps
Thanks
Kiran Kanetkar
Solution Architect |
|
| Back to top |
|
|
| Elmine |
| Posted: Mon Aug 18, 2003 2:20 am Post subject: |
|
|
Newbie
Joined: 12 Aug 2003
Posts: 9
Location: South Africa
|
Hi
This is the layout I'm getting in and that I have to write to a queue. The CH, DR and CT lines are repeated any number of times. I am now quite lost as to which physical format I should use. Can anybody please help.
FH0049720030519V
CHAZ4045092LE ROUX 5200431 1411934 000159522801
DR000159522801001010119270522 200304263743 ERYTHROCYTE SEDIMENTATION RATE00000155051376951 00100 F9894627
DR000159522801001020119270522 200304263755 BLOOD COUNT 00000650051376951 00100 F9894627
CT0001595228010000000100000002893000000000000000000000
CHAZ4046366COETZEE 5200431 2803623 000159522802
DR000159522802001010119520714 200304253743 ERYTHROCYTE SEDIMENTATION RATE00000155051309671 00100 F9892643
DR000159523797001030319591202F200304304182 CRP (C-REACTIVE PROT 000005130481717890 00100 L4080305
DR000159523797001020319591202F200304303975 R.CONORI IGM 000007430481717890 00100 L4080305
CT0001595237970000000050000001999000000000000000000000
BT0000007750000000960000325877800000000000000000000
FT00000270000000000400012758258000000319
Thanks
 |
|
| Back to top |
|
|
| Craig B |
| Posted: Mon Aug 18, 2003 2:57 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
Hi,
From your bitstream it can be seen that you the sections of the message can be identified. Ie You have a 'CH' in the bitstream which marks the start of the CH section etc. I would recommend using MRM-TDS but setting the Tagged Fixed Length data element separation instead of just fixed length. In the previous update I indicated that the problem with an unknown number of repetitions in Fixed Length is that the parser does not know where one field starts and the next one stops. However, Tagged Fixed Length does not suffer this problem, because the Tags identify where one element/structure starts, and then the fixed length definitions define where the element/structure ends. This way you get your repetitions on Tags, not on the Fixed length part. So the upshot is that Tagged Fixed length can support an unknown number of repetitions on both parsing/writing, where as Fixed Length cannot.
Hope this helps.
_________________
Regards
Craig |
|
| Back to top |
|
|
| Elmine |
| Posted: Mon Aug 18, 2003 6:21 am Post subject: |
|
|
Newbie
Joined: 12 Aug 2003
Posts: 9
Location: South Africa
|
Thanks, this definitely sounds like the format I'm looking for.
The problem is that we have never done this and I am battling to get all the settings right. The way I understand it, I have to declare compound types FH, CH, etc as Tagged Fixed Length and the Lenght of Tag = 2. Underneath each of these compound types I declare a number of elements with their different lengths, as declared in the file. As one of the element properties it wants me to declare a Tag value, of 2 long, for each element. This is the part I don't quite understand. The Tag should only be applicable to the compound type. How do I get past this, because it's inserting a tag in front of each element now.
Thanks a lot for all the help! |
|
| Back to top |
|
|
| Craig B |
| Posted: Mon Aug 18, 2003 6:43 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
Hi,
The overall parent will have a compound type that will be defined as Tagged Fixed length, and will have Length of Tag = 2. You will then have Compound elements within this main parent that will represent structures within your message such as FH, CH etc. These will be given the tags of 'FH' and 'CH' etc. However each of these compound elements is based on a compound type that has the lower level fixed length children. It is these compound types that have a Data Element Separation of Fixed length. These elements will have lengths associated with them and do not need Tags because they do not belong to a Tagged Fixed Length parent. It its the structures that will be repeating at the high level, and not the elements within the structures. Therefore you get the best of both worlds where you still parse fixed length structures, but the structures can repeat (unbounded) using the Tag.
So this is an example :
| Code: |
Message : myMessage (CT: DES=Tagged Fixed Length, LOT=2)
- FH Compound Element: Tag = FH (CT: DES=Fixed Length)
- Element1 (Length = 5)
- Element2 (Length = 5)
- CH Compound Element: Tag = CH (CT: DES=Fixed Length) {Repeat = Yes}
- Element3 (Length = 10)
- Element4 (Length = 20)
|
where
CT = Compound type MRM object is based on
DES = Data Element Separation
LOT = Length of Tag
Sorry I didnt make this clearer the first time.
_________________
Regards
Craig |
|
| Back to top |
|
|
| Elmine |
| Posted: Mon Aug 18, 2003 7:32 am Post subject: |
|
|
Newbie
Joined: 12 Aug 2003
Posts: 9
Location: South Africa
|
Hi Craig
I can't thank you enough! It's working now.
Thank you, thank you, thank you!!!
Regards
Elmine  |
|
| Back to top |
|
|
| Craig B |
| Posted: Mon Aug 18, 2003 7:58 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
No problem .... glad to be of help.
_________________
Regards
Craig |
|
| Back to top |
|
|
| Elmine |
| Posted: Tue Aug 19, 2003 1:10 am Post subject: |
|
|
Newbie
Joined: 12 Aug 2003
Posts: 9
Location: South Africa
|
Hi
I've encountered another obstacle. How do I get the output msg in the right format like in the example I posted?
If there's more than one CH record, it gives me all the CH records before the DR records. |
|
| Back to top |
|
|
| Craig B |
| Posted: Tue Aug 19, 2003 1:49 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
When you set the Tagged elements to be repeating in the main parent compound type, then this will group all these elements together as you have found. I wasn't aware that you wanted to maintain the same order on the output message that you had on the input message.
This will need a change in your MRM message that you have modelled. You need to change the Tagged Elements (such as DR and CH etc) so that they are not repeating. This is because you are going to put these Tagged elements into a repeating parent compound type/element. This way you have your blocks repeating instead of individual fields. You may think that not all the fields/structures appear in the block at anyone time. However, this is ok, because Tagged data element separation supports omitted elements, and so you do not have to have one of each of the DR, CH structures etc in each repeating block.
If you go for a surrounding nested compound type to repeat these fields, then this will not alter the fields references in your messageSet.
So off the top of my head (and I havent tried this), you need something like the following example :
| Code: |
Message : myMessage (CT: Composition=Sequence, DES=Tagged Fixed Length, LOT=2)
- NestedCompoundType (CT: DES=Tagged Fixed Length, LOT=2) {Repeat=Yes}
- FH Compound Element: Tag = FH (CT: DES=Fixed Length)
- Element1 (Length = 5)
- Element2 (Length = 5)
- CH Compound Element: Tag = CH (CT: DES=Fixed Length)
- Element3 (Length = 10)
- Element4 (Length = 20)
|
The myMessage main compound type needs to be of Type Composition sequence so that it can have a nested compound type inserted into it.
The one bit Im not 100% of at this stage is the myMessage compound type having a data element separation of Tagged Fixed length. If my understanding is correct then this parent will extract a tag of length 2, and then look to see if its children have this tag. This will only have one child and this is 'NestedCompoundType'. This does not have a tag, but is Tagged as well, so I hope that it will pass parsing control to nestedCompoundType who will look at its children to see if it has a tag of the current name. This should then carry on as before.
Give this a go and let me know how it goes. If I get chance later, I'll try out what I have suggested.
_________________
Regards
Craig |
|
| Back to top |
|
|
| Elmine |
| Posted: Tue Aug 19, 2003 10:22 pm Post subject: |
|
|
Newbie
Joined: 12 Aug 2003
Posts: 9
Location: South Africa
|
Hi
I tried this, but can't get it to work right. It removes the first two characters, CH & FH, because I declared these values as tags. I get a msg that looks like this in the debugger: 0049720030519VCH.
Then I get this error 'Mandatory element has no value or default value assigned', because it's not picking up CH.
 |
|
| Back to top |
|
|
| Craig B |
| Posted: Wed Aug 20, 2003 1:45 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
If you have the 'Mandatory element has no value or default value assigned' error message again, then this would be given on generating a new output message. Did the new message definition (with repeating parent) parse an input message successfully??
_________________
Regards
Craig |
|
| Back to top |
|
|
| Elmine |
| Posted: Wed Aug 20, 2003 4:58 am Post subject: |
|
|
Newbie
Joined: 12 Aug 2003
Posts: 9
Location: South Africa
|
Hi
No, I didn't get the message through succesfully.
I've got the order right, but I still have a problem with the tags. It removes the first 2 characters of each record because I declared it as tags. That means that everything is moved up 2 spaces, which is not correct. I want to remove the first field, that's being used as a tag, from each record, but the parser doesn't carry on with the message then. |
|
| Back to top |
|
|
| Craig B |
| Posted: Wed Aug 20, 2003 5:14 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
At this point I think it would be useful to see what you have defined. Would it be possible for you to send me a private message with your email address in, and then I can contact you for an export of your messageSet, flow and sample data??
_________________
Regards
Craig |
|
| Back to top |
|
|
|
|
