| Author |
Message
|
| jfrankman |
 Posted: Wed Sep 09, 2009 3:01 pm Post subject: XML to Fixed Length Conversion and Carriage Returns |
 |
|
Apprentice
Joined: 18 Jun 2009
Posts: 36
|
Is there a setting in the Message definition file that will place a carriage return after each instance of a xml element?
I have a flow that takes a message in XML format and I want to convert the same method to Fixed Length format (CWF) and write it to a file output node. However, it appears than when the conversion takes place and each XML element that represents a row is converted to fixed length, a carriage return and/or line feed is not placed between each fixed length record. Here is a sample of the XML:
| Code: |
<tns:FupRecords xmlns:tns="http://www.idfbins.com/ImageRightFUP" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.idfbins.com/ImageRightFUP ImageRightFUP.xsd ">
<tns:FupRecord>
<tns:Drawer>UNDR</tns:Drawer>
<tns:FileNumber>0111111101</tns:FileNumber>
<tns:FileName>John Doe</tns:FileName>
<tns:UserID>jdoe</tns:UserID>
<tns:Userkey1>123</tns:Userkey1>
<tns:Userdata1>1</tns:Userdata1>
<tns:Userdata2>2</tns:Userdata2>
<tns:Userdata3>3</tns:Userdata3>
<tns:Userdata4>4</tns:Userdata4>
<tns:Userdata5>5</tns:Userdata5>
<tns:NewFileNumber>0122222201</tns:NewFileNumber>
<tns:NewDrawer>UNDR</tns:NewDrawer>
<tns:FupStatus>0</tns:FupStatus>
<tns:StatusDate>20090101</tns:StatusDate>
<tns:UserIdLong>JDOE</tns:UserIdLong>
<tns:FileMark1>FM1</tns:FileMark1>
<tns:FileMark2>FM2</tns:FileMark2>
<tns:FileMark3>FM3</tns:FileMark3>
</tns:FupRecord>
<tns:FupRecord>
<tns:Drawer>UNDR</tns:Drawer>
<tns:FileNumber>033333301</tns:FileNumber>
<tns:FileName>John Doe</tns:FileName>
<tns:UserID>jdoe</tns:UserID>
<tns:Userkey1>123</tns:Userkey1>
<tns:Userdata1>1</tns:Userdata1>
<tns:Userdata2>2</tns:Userdata2>
<tns:Userdata3>3</tns:Userdata3>
<tns:Userdata4>4</tns:Userdata4>
<tns:Userdata5>5</tns:Userdata5>
<tns:NewFileNumber>0144444401</tns:NewFileNumber>
<tns:NewDrawer>UNDR</tns:NewDrawer>
<tns:FupStatus>0</tns:FupStatus>
<tns:StatusDate>20090101</tns:StatusDate>
<tns:UserIdLong>JDOE</tns:UserIdLong>
<tns:FileMark1>FM1</tns:FileMark1>
<tns:FileMark2>FM2</tns:FileMark2>
<tns:FileMark3>FM3</tns:FileMark3>
</tns:FupRecord>
</tns:FupRecords> |
For each FupRecord element, I want a carriage return to appear when the xml is converted to fixed length records. So the above XML should produce two fixed length records in the file output node's text file.
The flow consists of:
MQInput -> Compute -> FileOutput
In the compute node I only have two lines of code:
| Code: |
SET OutputRoot = InputRoot;
Set OutputRoot.Properties.MessageFormat='Binary1'; |
|
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Wed Sep 09, 2009 9:28 pm Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
first of all you have to model the fixed lenth format..
make the parent folder to All Elements Delimited <CR><LF>
make the child folder to Fixed Length
Keep the elements over here
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Sep 10, 2009 1:35 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| it appears than when the conversion takes place and each XML element that represents a row is converted to fixed length, a carriage return and/or line feed is not placed between each fixed length record. |
It appears that you think that all fixed-length formats have line feeds and/or carriage returns between the records.
Think about it. Is that a reasonable assumption? Does the CWF format offer you any settings to control the linefeed character? What do the docs say about parsers and physical formats?
Gaya3: What were you trying to accomplish with your post?
 |
|
| Back to top |
|
|
| jfrankman |
| Posted: Thu Sep 10, 2009 3:32 pm Post subject: |
|
|
Apprentice
Joined: 18 Jun 2009
Posts: 36
|
Kimbert,
Thank you for your kind reply. As a beginner and part time MB developer, I can't believe I had the audacity to ask a question on a forum! You remarks are most helpful.
I especially like this:
| Quote: |
It appears that you think that all fixed-length formats have line feeds and/or carriage returns between the records.
Think about it. Is that a reasonable assumption? |
Even though 99% of all fixed length files I have come across and line feeds/carriage returns as delimiters, how dare I make such an assumption. I mean carriage returns to delimit fixed length records????, Sheesh!
I also liked this:
| Quote: |
| Does the CWF format offer you any settings to control the linefeed character? |
Good job re-stating the essence of my question and yet providing no answer.
But the crown jewel has to be this:
| Quote: |
| What do the docs say about parsers and physical formats? |
Wait a minute, do you mean to say there is documentation? Oh yeah!!! Of course I do remember reading through the documentation on and off for a couple days to try to figure out this problem but perhaps I need to take a 1 month sabbatical to read every single document rather than ask a simple question of how to convert records in xml into fixed length records.
I asked the question the best way I knew how, but that was clearly not good enough for the likes of people like you. Nice job taking your time to put me down rather than just ignoring the post or actually answering the question. I will make sure I memorize all documentation and become omniscient before I dare enter this forum from now on. I am clearly not worthy.
Is there a forum I can go to for regular people who have not worked several years developing with the message broker and have not read every inch of documentation? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Sep 10, 2009 3:48 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
@ jfrankman
What's up with the tude? Your post did not specify whether or not you had read any documentation. And 99% of the files you came across may have had a CRLF as record delimiter.... some files have a format assigned and no line delimiter... Anyways the doc says how to model the Record delimiter...
Have you looked into MRM and TDS parsers? Have you looked at the possibilities to have a record delimiter in the MRM CWF parser while looking at your message set? And typically the Full XML document represents one record. If you have 1 record per XML element you should maybe model that, and use propagate for each of the XML elements...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jfrankman |
| Posted: Thu Sep 10, 2009 3:57 pm Post subject: |
|
|
Apprentice
Joined: 18 Jun 2009
Posts: 36
|
| Quote: |
| And typically the Full XML document represents one record |
Thank you. Propogating the messages was what I planned to do initially, but I thought that maybe there was slick way to change the MessageFormat from XML to CWF and specify in the CWF that each message ends in a line feed. But, I will loop through the XML file and I Propagate each XML record separately to the file output node one at a time rather than the entire XML file.
Please excuse my previous rant and thanks for the help. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Sep 10, 2009 5:24 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jfrankman wrote: |
| Is there a forum I can go to for regular people who have not worked several years developing with the message broker and have not read every inch of documentation? |
Yes, but no-one there will be able to help because they're too inexperienced and don't know the product or it's documentation. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jfrankman |
| Posted: Thu Sep 10, 2009 7:32 pm Post subject: |
|
|
Apprentice
Joined: 18 Jun 2009
Posts: 36
|
From fjb_saper's comments I was able to get this going. I am using the propogate in a compute node to split the single xml message containing several records into separate messages sent to the FileOutput node:
| Code: |
CREATE COMPUTE MODULE SimpleFup_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
DECLARE i INTEGER 1;
DECLARE count INTEGER;
SET count = CARDINALITY(InputRoot.MRM.ns2:FupRecord[]);
WHILE i <= count DO
CALL CopyMessageHeaders();
Set OutputLocalEnvironment.Destination.File.Directory='c:\work';
Set OutputLocalEnvironment.Destination.File.Name='zzfup.txt';
Set OutputRoot.Properties.MessageFormat='Binary1';
SET OutputRoot.MRM.ns2:FupRecord = InputRoot.MRM.ns2:FupRecord[i];
PROPAGATE TO TERMINAL 'out1';
SET i = i+1;
END WHILE;
Set OutputLocalEnvironment.Destination.File.Directory='c:\work';
Set OutputLocalEnvironment.Destination.File.Name='zzfup.txt';
RETURN TRUE;
END;
CREATE PROCEDURE CopyMessageHeaders() BEGIN
DECLARE I INTEGER 1;
DECLARE J INTEGER;
SET J = CARDINALITY(InputRoot.*[]);
WHILE I < J DO
SET OutputRoot.*[I] = InputRoot.*[I];
SET I = I + 1;
END WHILE;
END;
CREATE PROCEDURE CopyEntireMessage() BEGIN
SET OutputRoot = InputRoot;
END;
END MODULE; |
Thanks. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Sep 11, 2009 2:49 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Hi jfrankman,
Looks as if my reply conveyed completely the wrong impression ( so my fault, not yours ). I was trying to teach you how to fish, rather than giving you the specific fish that you asked for. Sorry if it came across as a put-down.
If you want your records delimited by line feeds then you need the TDS parser. CWF is for fixed-length formats ( typically COBOL or C ). It is quite common for CWF formats not to contain linefeeds between the records.
Anyway - glad you got a working solution. You could, if you wanted to, simplify it quite a lot by using TDS to insert the linefeeds for you. You wouldn't need that loop/propagate in your ESQL then. |
|
| Back to top |
|
|
| jfrankman |
| Posted: Fri Sep 11, 2009 8:57 am Post subject: |
|
|
Apprentice
Joined: 18 Jun 2009
Posts: 36
|
Kimbert,
Sorry for my abrupt rant. I should not have jumped to conclusions so quickly. Thanks for the understanding, I was working late and was tired. I should not have responded the way I did.
Anyway, even though it is working with the loop/propagate, I would like to get this to work with TDS. I looked at the CSV sample as a model. I created a new tds physical format. On my repeating element where minoccurs =0 and maxoccurs =-1 I added <CR><LF> as the repeating element delimiter. However, the output file still has both records as a single line. Is there something else besides this that I need to do to make this work? Should the "Record Definition" property in the FileOutput node be "Record is Delimited data" or something else? |
|
| Back to top |
|
|
|
|
