| Author |
Message
|
| 6thelement |
 Posted: Wed Sep 01, 2010 1:09 pm Post subject: Repeating Fixed Length Structure |
 |
|
Apprentice
Joined: 07 Nov 2008
Posts: 30
|
Hi,
Here is my problem statement:
Source Data:SAP IDOC
Destination Data:Pipe Delimited Flat File
Issue: I need to model the input message set for the IDOC structure as described next.
SAP IDOC contains multiple segments where each segment can repeat n number of times. Some of the segments are grouped together (kind of Parent-Child relationship) which would repeat again n number of times.
Each segment has fixed length of 10 bytes and first 5 bytes of the each segment contains seg_id to denote the particular segment.
SAP has sent C Header File which is imported as MRM-CWF in Message Set of Message Broker and it contains multi-part. I saw it somewhere in this forum that its not possible to design a message model in CWF where repeating fixed length structures are there.
To give an example: Say the IDOC contains four Segments.
TAG01ABCDETAG12FGHIJTAG22KLMNOTAG03PQRST
For readability:
TAG01ABCDE
TAG12FGHIJ
TAG22KLMNO
TAG03PQRST
Each of these segments can repeat multiple times. Also the whole container structure can repeat multiple times and there is no delimiter between two container structures. Also SE_12 and SE_22 are contained in one group with repeatation of 0 to -1.
To depict in example"
TAG01ABCDETAG12FGHIJTAG22KLMNOTAG03PQRSTTAG01UVQWETAG12UVWXYTAG22ZWSAQTAG12XYZABTAG22CDEFGTAG22BATMATAG03EFGHITAG03JKLMN
For Readability:
TAG01ABCDE
TAG12FGHIJ
TAG22KLMNO
TAG03PQRST
TAG01UVQWE
TAG12UVWXY
TAG22ZWECB
TAG12XYZAB
TAG22CDEFG
TAG22BATMA
TAG03EFGHI
TAG03JKLMN
Here the container structure repeats twice. First container structure contains single instance of all three segments. Second container structure contains single instance of TAG01, three instances of TAG12/TAG22 group and two instances of TAG03.
I changed the message set to parse the IDOC as MRM-TDS. I understand I need to use Tagged Fixed Length here. But I am not getting how do I model the message set and what properties have to be set.
Here is the Message Set I thought of:
CONTAINERCOMPTYPE: - Tagged Fixed Length with Tag Length 5
TAG01COMPTYPE - Tagged Fixed Length with Tag Length 5 and Data
Length 5
TAG12COMPTYPE - Tagged Fixed Length with Tag Length 5 and Data
Length 5
TAG22COMPTYPE - Tagged Fixed Length with Tag Length 5 and Data
Length 5
TAG03COMPTYPE - Tagged Fixed Length with Tag Length 5 and Data
Length 5
Please let me know whether I am clear in the problem description. Would need your help in the solving this issue of message modeling.
Plz let me know what other info is required.
Thanks.. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Sep 01, 2010 2:13 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
you want to be using the SAP Adapter. If you don't care about the IDOC content and need do no transformation you can pass it as a BLOB.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| 6thelement |
| Posted: Wed Sep 01, 2010 8:48 pm Post subject: |
|
|
Apprentice
Joined: 07 Nov 2008
Posts: 30
|
Hi,
Forgot to mention. SAP Adapter is used here and thats how I get these fixed length SAP IDOC data into Message Broker.
We are doing the transformation here into pipe delimited data as I mentioned in my post and so handling BLOB data directly may not be a good idea.
Thanks... |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Sep 02, 2010 1:18 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| I saw it somewhere in this forum that its not possible to design a message model in CWF where repeating fixed length structures are there. |
CWF can handle repeating fixed-length structures, but only if the number of repeats is fixed or is contained in another field in the message.
CWF cannot handle maxOccurs=-1 unless the element/group is the last item in the message.
| Quote: |
| I changed the message set to parse the IDOC as MRM-TDS. I understand I need to use Tagged Fixed Length here. |
Correct on both counts. TDS is required for the maxOccurs=-1.
I suggest a logical model that looks like this:
| Code: |
element name='message'
complexType DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='container' maxOccurs='unbounded'
complexType DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='TAG01' Tag='TAG01' length='5'
sequence DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='TAG11' Tag='TAG11' length='5'
element name='TAG12' Tag='TAG12' length='5'
element name='TAG03' Tag='TAG03' length='5' |
|
|
| Back to top |
|
|
| 6thelement |
| Posted: Thu Sep 02, 2010 2:20 am Post subject: |
|
|
Apprentice
Joined: 07 Nov 2008
Posts: 30
|
| Quote: |
element name='message'
complexType DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='container' maxOccurs='unbounded'
complexType DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='TAG01' Tag='TAG01' length='5'
sequence DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='TAG11' Tag='TAG11' length='5'
element name='TAG12' Tag='TAG12' length='5'
element name='TAG03' Tag='TAG03' length='5' |
May be to simplify the problem statement, I gave a example which didnt depict my problem properly.
[The actual segments are of 1030 bytes each and each segment structure is different contaning varying no. of fields. The first field in the segment is common across all the segments in terms length of 30 bytes and that can be used as a Tag. Rest of the 1000 fields contain the data and this data has to be parsed by Message Broker too]
Going back to the example, I am bit confused what needs to be set within Container.
Each segment is of 10 bytes, but I am seeing length = 5 for each segment. Also, after the tag is identified, the data has to be parsed to successfully also. How will that happen?
* when you say sequence, is it one more complex type you want me to create to represent the sub-group within main container.
*e.g.[element name='TAG01' Tag='TAG01' length='5' ] -> This element itself is a type of complextype(Fixed Length, 10) containing varying length fields other than the Tag and all the fields has to be identified too. So I wouldn't get this Tag and Length Option in TAG01 element under Container. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Sep 02, 2010 3:07 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Tip : Please use [code] tags instead of [quote] tags when posting indented data. It makes it much more readable.
| Quote: |
I am bit confused what needs to be set within Container.
Each segment is of 10 bytes, but I am seeing length = 5 for each segment |
I was assuming that you had a 5-character tag followed by a 5-character value. Hence I set TagLength='5' and length='5'.
| Quote: |
| Also, after the tag is identified, the data has to be parsed to successfully also. How will that happen? |
The tag is not part of the data, and it will not appear in the message tree. After an element's tag has been found in the bitstream, the element's value ( if it is a simple element ) or content ( if it is a complex element ) will be parsed.
| Quote: |
* when you say sequence, is it one more complex type you want me to create to represent the sub-group within main container.
|
No - it is a sequence group. You can add a sequence group ( or a choice group ) as a child of any complex element ( right-click and select 'Add sequence'. The group does not cause a new level to be created in the message tree - the contents of the group will be siblings of the other members of the element.
i.e. in the message tree, TAG11 and TAG12 will have the same parent as TAG01 |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Sep 02, 2010 7:43 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I am very confused. Most SAP IDOC segments do not repeat infinitely. They have a minimum number of occurrences and a max number of occurrences.
This is all part of the definition of the IDOC in SAP.
What is the problem you are facing there? Are you repeating the number of Idocs ad infinitum?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| 6thelement |
| Posted: Sat Sep 04, 2010 10:21 am Post subject: |
|
|
Apprentice
Joined: 07 Nov 2008
Posts: 30
|
@Kimbert
Your suggestion is working. I created sequence group and its parsing successfully. But another problem I am facing is when the children are multi-level (Like one Sequence contains another sequence, so on so forth).
e.g.
| Code: |
element name='message'
complexType DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='container' maxOccurs='unbounded'
complexType DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='TAG01' Tag='TAG01' length='5'
sequence DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='TAG11' Tag='TAG11' length='5'
sequence DataElementSeparation='Tagged Fixed Length' TagLength='5'
element name='TAG1A' Tag='TAG1A' length='5'
element name='TAG1B' Tag='TAG1B' length='5'
element name='TAG12' Tag='TAG12' length='5'
element name='TAG03' Tag='TAG03' length='5' |
Its failing while accessing TAG1A elements.
@fjb_saper
I understand all the IDOCs are fixed length structures only. But what I understood from SAP guys is the segments can repeat indefinite number of times.  ..So I had to make the maxOccurs = -1.
On a second thought: In such a scenario of multi-level, multi-group IDOC segments say if I make the maxOccurs=99999, would the CWF thing will work? Will the CWF structure will be able to identify the segments properly? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Sep 04, 2010 10:47 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
What you really want to do is use the SAP adapter to model the IDOC.
It will retrieve the IDOC's definition from the SAP repository and create the matching message set to be used. I would expect this to be some kind of TDS structure where each Segment is identified by it's segment name.
You can also look at each individual IDOC as an EDI_DC40 followed by an indetermined number of EDI_DD40's. You can model the EDI_DC40 and EDI_DD40 by the segment name...
The IDOC name and EXTENSION (see EDI_DC40) determines how many segments of which type and at which level you may encounter in the document.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| 6thelement |
| Posted: Sat Sep 04, 2010 11:01 am Post subject: |
|
|
Apprentice
Joined: 07 Nov 2008
Posts: 30
|
One more piece of information. We are using the MQLink Adapter here.  . So it wont give me the message set automatically. I need to import the C Header file and then construct the message definition file manually based on the order of segments and repeatations etc. Am I missing something?
Earlier I have used mySAP adapter and also the new inbuilt SAP node. |
|
| Back to top |
|
|
| kimbert |
| Posted: Sat Sep 04, 2010 11:18 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| I created sequence group and its parsing successfully. But another problem I am facing is when the children are multi-level (Like one Sequence contains another sequence, so on so forth). |
The problem could be anywhere - it's really hard for somebody else to diagnose the problem for you.
I have a standard method for diagnosing all ( non-trivial ) TDS parsing problems.
- insert a Trace node after the input node with pattern {$Root}
- take a debug-leveluser trace
- put a message through the flow
- read the user trace especially the TDS parser's entries.
Before you do that, you might like to check that maxOccurs is set correctly on the sequence group.
You may need to get creative. If it's not working, you can shorten the input message ( and the message model, of course) until it *is* working, and then move forward from there. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Sep 04, 2010 6:00 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| 6thelement wrote: |
One more piece of information. We are using the MQLink Adapter here. . So it wont give me the message set automatically. I need to import the C Header file and then construct the message definition file manually based on the order of segments and repeatations etc. Am I missing something?
Earlier I have used mySAP adapter and also the new inbuilt SAP node. |
We receive Idocs in MQLink format too. You parse them in the IDOC domain.
Download the IDOC format from SAP (c structure or COBOL copy book). You may have to add the filler for each segment end till the 1000 char for segment data. Make sure the each segment name is equal to the a corresponding Message Name. Have all segments in the same message set.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| 6thelement |
| Posted: Sat Sep 04, 2010 9:10 pm Post subject: |
|
|
Apprentice
Joined: 07 Nov 2008
Posts: 30
|
@fjb_saper
Oh.yes. We have done that for few projects as you described where
* C Header is imported to Message Set as MRM/Binary
* New Message is created is creating reference to edi_dc40 messages and number of edi_dd40 messages
* All the fillers are as required, but the repeating segments are always the last segment in the message
But never faced a situation where we need to have number multiple segments each repeating indefinite times and having multi-level grouping upto 5 level down.
@kimbert
yes. I do follow that method. But this time I followed simple logic that it may not work out because we have upto say 5 levels and how sequence group would work there. Now I will put a trace and will debug it. If not solved, will let you know guys. |
|
| Back to top |
|
|
| kimbert |
| Posted: Sun Sep 05, 2010 9:15 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| it may not work out because we have upto say 5 levels and how sequence group would work there. |
I don't blame you for wondering about it, but the facts are as follows:
- a sequence group is not restricted in any way. Anything that you can do using a complex element can be done using a sequence group.
- the number of levels does not matter. If the format is parseable, then ( with a couple of exceptions ) the TDS parser can handle it. |
|
| Back to top |
|
|
|
|
