| Author |
Message
|
| giorginus80 |
 Posted: Tue Jul 08, 2008 1:51 am Post subject: Fixed length to XML |
 |
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
Hi,
I'm having troubles transforming a flat fixed file to XML. I have message definition, and I can parse single records, but I can't know how to determine the GROUP in the file. I mean I have some record in the file (fixed) and then 1 record (a report, fixed length too, i recognize it with regular expression), and this is a group. How can I determine the end of a group in the fixed length file format?
For example in the fixed length flat file I have 2 kind of record ( both 25 bytes length I recognize one from other beacuse the first end with '999' and could be N and the second starts with '999' and could be only one)
GIORGIO FEDERICI 88928
PAOLO CRETI 83310
FEDERICO CILICHI 23322
78473UURIIRURJKNFDJFDJFDI
ROBERTA LIPARI 98982
STEFANO DESANTICST82929
GIORGIANAIPILACCIN 92992
HYUADAOIUUUUIU9823792JDJD
Last edited by giorginus80 on Tue Jul 08, 2008 5:24 am; edited 1 time in total |
|
| Back to top |
|
 |
| kimbert |
| Posted: Tue Jul 08, 2008 3:26 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Your example message does not contain '999'. Please supply an example message, and state any special parsing rules. Supply as much detail as possible. I should be able to help after that.
As a general rule, it is not possible to make general rules about parsing problems. Every case is different, unless you are using a well-known message format. |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Tue Jul 08, 2008 4:31 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
yes sorry, you are right my example was
GIORGIO FEDERICI 88999
PAOLO CRETI 83999
FEDERICO CILICHI 23999
99973UURIIRURJKNFDJFDJFDI
ROBERTA LIPARI 98999
STEFANO DESANTICST82999
GIORGIANAIPILACCIN 92999
999ADAOIUUUUIU9823792JDJD
and I would like the output with this:
| Code: |
<?xml version="1.0"?>
<test:FixedFile xmlns:test="http://www.test.it" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<record>
<person>
<name>GIORGIO </name>
<surname>FEDERICI </surname>
<code>88999</code>
</person>
<person>
<name>PAOLO </name>
<surname>CRETI </surname>
<code>83999</code>
</person>
<person>
<name>FEDERICO </name>
<surname>CILICHI </surname>
<code>23999</code>
</person>
<resume>
<sector>99973</sector>
<param1>UURIIRURJKNFDJF</param1>
<param2>DJFDI</param2>
</resume>
</record>
<record>
<person>
<name>ROBERTA </name>
<surname>LIPARI </surname>
<code>98999</code>
</person>
<person>
<name>STEFANO </name>
<surname>DESANTICST</surname>
<code>82999</code>
</person>
<person>
<name>GIORGIANAI</name>
<surname>PILACCIN </surname>
<code>92999</code>
</person>
<resume>
<sector>999AD</sector>
<param1>AOIUUUUIU982379</param1>
<param2>2JDJD</param2>
</resume>
</record>
</test:FixedFile>
|
I cannot group person 1 to -1 and resume 1 to 1 in the tag record. I use WMB 6.1.0.2
Thanks in advance
MY mxsd is:
| Code: |
<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:test="http://www.test.it" targetNamespace="http://www.test.it">
<xsd:complexType name="personType">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsStructRep groupTerminator="" messageSetDefaultRep="Text1"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:sequence>
<xsd:element name="name" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="10" lengthUnits="Bytes" messageSetDefaultRep="Text1" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:element name="surname" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="10" lengthUnits="Bytes" messageSetDefaultRep="Text1" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:element name="code" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="5" lengthUnits="Bytes" messageSetDefaultRep="Text1" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="resumeType">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsStructRep dataElementSeparation="FixedLength" groupTerminator="" messageSetDefaultRep="Text1"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:sequence>
<xsd:element name="sector" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="5" lengthUnits="Bytes" messageSetDefaultRep="Text1" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:element name="param1" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="15" lengthUnits="Bytes" messageSetDefaultRep="Text1" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:element name="param2" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="5" lengthUnits="Bytes" messageSetDefaultRep="Text1" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="recordsType">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsStructRep dataElementSeparation="UseDataPattern" groupTerminator="<CR><LF>" messageSetDefaultRep="Text1"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:sequence maxOccurs="1" minOccurs="1">
<xsd:element form="qualified" maxOccurs="unbounded" minOccurs="0" name="person" type="test:personType">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep dataPattern="[a-zA-Z0-9\s]{22}[9]{3}" messageSetDefaultRep="Text1" precision="-1"/>
<tdsInclRep dataPattern="[a-zA-Z0-9\s]{22}[9]{3}" messageSetDefaultRep="Text1" repeatingElementDelimiter=""/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:element form="qualified" minOccurs="1" name="resume" type="test:resumeType">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep dataPattern="[9]{3}[a-zA-Z0-9\s]{22}" messageSetDefaultRep="Text1" precision="-1"/>
<tdsInclRep dataPattern="[9]{3}[a-zA-Z0-9\s]{22}" messageSetDefaultRep="Text1" repeatingElementDelimiter=""/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="fileType">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsStructRep dataElementSeparation="FixedLength" delimiter="" groupTerminator="" messageSetDefaultRep="Text1"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:sequence>
<xsd:element maxOccurs="unbounded" name="record" type="test:recordsType">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep messageSetDefaultRep="Text1" precision="-1"/>
<tdsInclRep messageSetDefaultRep="Text1" repeatingElementDelimiter=""/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
<xsd:element name="FixedFile" type="test:fileType">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<MRMessage messageDefinition="/0/FixedFile;XSDElementDeclaration$MRObject"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:schema>
|
if you need I uploaded to IBM my example with Flow and MessageSet in a zip the url is http://www-128.ibm.com/developerworks/forums/servlet/JiveServlet/download/281-213252-14106240-326098/FixLengthFlow.zip |
|
| Back to top |
|
|
| sridhsri |
| Posted: Tue Jul 08, 2008 5:34 am Post subject: |
|
|
Master
Joined: 19 Jun 2008
Posts: 297
|
| Perhaps you can use data pattern to distinguish between "person" tag and "resume" tags. I am guessing that records that begin with alphabets are "person" tags and ones the begin with numbers are "resume" |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jul 08, 2008 5:39 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I can see why you are having a problem. The message format is not an easy one to parse!
The best way to handle this is to parse the entire 'record' element using data patterns. One data pattern for the 'person' element and another for the 'resume' element. Something like this:
| Code: |
Element name="Record", maxOccurs="-1"
ComplexType DataElementSeparation="Use Data Pattern"
element name="person", maxOccurs="-1", DataPattern="[^9][^\s]*\s"
element name="resume", DataPattern="9{3}[A-Za-z0-9]*\s"
|
|
|
| Back to top |
|
|
| giorginus80 |
| Posted: Tue Jul 08, 2008 5:41 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| sridhsri wrote: |
| Perhaps you can use data pattern to distinguish between "person" tag and "resume" tags. I am guessing that records that begin with alphabets are "person" tags and ones the begin with numbers are "resume" |
I can distinguish them, I have the result, but not the group
The wrong result in my code is:
| Code: |
<?xml version="1.0"?>
<test:FixedFile xmlns:test="http://www.test.it" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<record>
<person>
<name>GIORGIO </name>
<surname>FEDERICI </surname>
<code>88999</code>
</person>
</record>
<record>
<person>
<name>PAOLO </name>
<surname>CRETI </surname>
<code>83999</code>
</person>
</record>
<record>
<person>
<name>FEDERICO </name>
<surname>CILICHI </surname>
<code>23999</code>
</person>
</record>
<record>
<resume>
<sector>99973</sector>
<param1>UURIIRURJKNFDJF</param1>
<param2>DJFDI</param2>
</resume>
</record>
<record>
<person>
<name>ROBERTA </name>
<surname>LIPARI </surname>
<code>98999</code>
</person>
</record>
<record>
<person>
<name>STEFANO </name>
<surname>DESANTICST</surname>
<code>82999</code>
</person>
</record>
<record>
<person>
<name>GIORGIANAI</name>
<surname>PILACCIN </surname>
<code>92999</code>
</person>
</record>
<record>
<resume>
<sector>999AD</sector>
<param1>AOIUUUUIU982379</param1>
<param2>2JDJD</param2>
</resume>
</record>
</test:FixedFile>
|
meanwhile I want the person (1 to -1) and 1 resume in the record
| Code: |
<?xml version="1.0"?>
<test:FixedFile xmlns:test="http://www.test.it" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<record>
<person>
<name>GIORGIO </name>
<surname>FEDERICI </surname>
<code>88999</code>
</person>
<person>
<name>PAOLO </name>
<surname>CRETI </surname>
<code>83999</code>
</person>
<person>
<name>FEDERICO </name>
<surname>CILICHI </surname>
<code>23999</code>
</person>
<resume>
<sector>99973</sector>
<param1>UURIIRURJKNFDJF</param1>
<param2>DJFDI</param2>
</resume>
</record>
<record>
<person>
<name>ROBERTA </name>
<surname>LIPARI </surname>
<code>98999</code>
</person>
<person>
<name>STEFANO </name>
<surname>DESANTICST</surname>
<code>82999</code>
</person>
<person>
<name>GIORGIANAI</name>
<surname>PILACCIN </surname>
<code>92999</code>
</person>
<resume>
<sector>999AD</sector>
<param1>AOIUUUUIU982379</param1>
<param2>2JDJD</param2>
</resume>
</record>
</test:FixedFile>
|
|
|
| Back to top |
|
|
| giorginus80 |
| Posted: Tue Jul 08, 2008 5:54 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| kimbert wrote: |
I can see why you are having a problem. The message format is not an easy one to parse!
The best way to handle this is to parse the entire 'record' element using data patterns. One data pattern for the 'person' element and another for the 'resume' element. Something like this:
| Code: |
Element name="Record", maxOccurs="-1"
ComplexType DataElementSeparation="Use Data Pattern"
element name="person", maxOccurs="-1", DataPattern="[^9][^\s]*\s"
element name="resume", DataPattern="9{3}[A-Za-z0-9]*\s"
|
|
I used the data pattern, and I can distinguish person from resume, but I need to group in the record more than one person and only one resume, when there is the resume it means is the last thing to put in the record tag, so let's start another record tag, and I can't know how to tell the messageset to group in the record 1 to -1 person and 1 to 1 resume
if you can, you could see my zip attached in the ibm site. There is all my example.
http://www-128.ibm.com/developerworks/forums/servlet/JiveServlet/download/281-213252-14106240-326098/FixLengthFlow.zip |
|
| Back to top |
|
|
| sridhsri |
| Posted: Tue Jul 08, 2008 6:37 am Post subject: |
|
|
Master
Joined: 19 Jun 2008
Posts: 297
|
Tim,
I have a dumb question - but would it be a good idea to map the incoming fixed length message to the XML message instead of modeling it ? We could perhaps use Mapping/ESQL to create the XML message. |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Tue Jul 08, 2008 6:41 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| sridhsri wrote: |
Tim,
I have a dumb question - but would it be a good idea to map the incoming fixed length message to the XML message instead of modeling it ? We could perhaps use Mapping/ESQL to create the XML message. |
I don't know how to do it....but is it possible that in messageSet properties can't I create more 'record' tag with 1 or more 'person' and 1 'resume' ? |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jul 08, 2008 7:09 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
sridhsri: In general, it's not a good idea to use ESQL to parse the input message. It makes the flow logic dependent on the input message format. One of the main strengths of WMB is that the message can be processed as a logical structure independent of its physical (on-the-wire) format.
giorginus80: I must be missing something here. My suggested solution has record with maxOccurs=-1, and person with maxOccurs=-1, and resume with maxOccurs=1 ( implicitly ). That should be exactly what you are asking for. Am I being thick-headed here? |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Tue Jul 08, 2008 7:13 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| kimbert wrote: |
sridhsri: In general, it's not a good idea to use ESQL to parse the input message. It makes the flow logic dependent on the input message format. One of the main strengths of WMB is that the message can be processed as a logical structure independent of its physical (on-the-wire) format.
giorginus80: I must be missing something here. My suggested solution has record with maxOccurs=-1, and person with maxOccurs=-1, and resume with maxOccurs=1 ( implicitly ). That should be exactly what you are asking for. Am I being thick-headed here? |
Yes if you can download my zip file, it's right you wrote, it's all ok, but the message isn't parsed like I want! Please help me. it's very important! I can't build the record |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Jul 09, 2008 12:26 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| if you can download my zip file |
Sorry - I'm not a consultant, I'm a developer who likes to help out. You tell me what the problem is, and I will suggest a solution. |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Wed Jul 09, 2008 12:44 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| kimbert wrote: |
| Quote: |
| if you can download my zip file |
Sorry - I'm not a consultant, I'm a developer who likes to help out. You tell me what the problem is, and I will suggest a solution. |
Yes sorry,
I can't know how to group the record, how can I tell with messageSet that one or more 'person' and one 'resume' are in one 'record' then start another 'record' ?
thanks in advance |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Jul 09, 2008 12:47 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Please insert a Trace node and post the output that you are getting. Or if you are getting error messages, post those.
If you're on v6.1, please also take a debug-level user trace of the parse and post that. |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Wed Jul 09, 2008 12:51 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| kimbert wrote: |
Please insert a Trace node and post the output that you are getting. Or if you are getting error messages, post those.
If you're on v6.1, please also take a debug-level user trace of the parse and post that. |
please don't kill me....how can I trace the parse? I'm using 6.1
sorry for me! |
|
| Back to top |
|
|
|
|
