| Author |
Message
|
| diamondiona |
 Posted: Thu Jun 19, 2014 6:57 pm Post subject: TDS Header |
 |
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
Hi everyone,
My business case is:
1. FileInputNode parses the following input messages using xmlnsc parser:
<person>
<firstname>firstname1</firstname>
<lastname>lastname1</lastname>
</person>
<person>
<firstname>firstname2</firstname>
<lastname>lastname2</lastname>
</person>
2. ESQL compute node transform this message to CSV format using MRM parser. Generated file looks like this:
firstname1,lastname1
firstname2,lastname2
My question is: is it possible to add header information to generated CSV file? like
header --> firstname,secondname
firstname1,secondname1
firstname2,secondname2
Thanks a lot for your help!
Diamond |
|
| Back to top |
|
 |
| aggarwal.intouch |
| Posted: Thu Jun 19, 2014 10:46 pm Post subject: |
|
|
Acolyte
Joined: 30 May 2011
Posts: 56
Location: India
|
| You can add these header constant values while creating your message set. Keep occurrence of these elements as 1..1 only. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Jun 20, 2014 2:35 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Why not just add the header row into the message tree?
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| diamondiona |
| Posted: Fri Jun 20, 2014 10:18 am Post subject: |
|
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
Can you guys please give me more information how to do that? By the way, my version is 7.0
Truly Appreciate! |
|
| Back to top |
|
|
| diamondiona |
| Posted: Fri Jun 20, 2014 6:43 pm Post subject: |
|
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
| Can you guys please give me an example? Thanks a lot! |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Jun 20, 2014 10:42 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
How would you add an item to a message tree?
We really like people to try things for themselves. You learn a lot more that way. Don't be afraid of saying 'I tried this .... It didn't work'.
You won't loase any face here. In fact, you will gain Kudos for being willing to try things for yourself.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| diamondiona |
| Posted: Sat Jun 21, 2014 3:06 pm Post subject: |
|
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
I did some tests but failed to generate the header and output file is always empty. I didn't find any way to attach files. So I just copied and pasted in here. Sorry for the inconvenience and thanks for help!
Message Flow Project
| Code: |
.project
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>Test</name>
<comment></comment>
<projects>
<project>TestMsg</project>
</projects>
<buildSpec>
<buildCommand>
<name>com.ibm.etools.mft.esql.lang.esqllangbuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>com.ibm.etools.mft.mapping.builder.mappingbuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>com.ibm.etools.mft.flow.msgflowxsltbuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>com.ibm.etools.mft.flow.msgflowbuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>com.ibm.etools.mft.bar.ext.barbuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>com.ibm.etools.mft.flow.messageflownature</nature>
<nature>com.ibm.etools.mft.bar.ext.barnature</nature>
</natures>
</projectDescription> |
in.xml
| Code: |
<person>
<firstname>firstname</firstname>
<lastname>lastname</lastname>
</person>
<person>
<firstname>firstname</firstname>
<lastname>lastname</lastname>
</person> |
test.msgflow
| Code: |
<?xml version="1.0" encoding="UTF-8"?>
<ecore:EPackage xmi:version="2.0"
xmlns:xmi="http://www.omg.org/XMI" xmlns:ComIbmCompute.msgnode="ComIbmCompute.msgnode" xmlns:ComIbmFileInput.msgnode="ComIbmFileInput.msgnode" xmlns:ComIbmFileOutput.msgnode="ComIbmFileOutput.msgnode" xmlns:ecore="http://www.eclipse.org/emf/2002/Ecore" xmlns:eflow="http://www.ibm.com/wbi/2005/eflow" xmlns:utility="http://www.ibm.com/wbi/2005/eflow_utility" nsURI="Test.msgflow"
nsPrefix="Test.msgflow">
<eClassifiers xmi:type="eflow:FCMComposite" name="FCMComposite_1">
<eSuperTypes href="http://www.ibm.com/wbi/2005/eflow#//FCMBlock"/>
<translation xmi:type="utility:TranslatableString" key="Test" bundleName="Test" pluginId="Test"/>
<colorGraphic16 xmi:type="utility:GIFFileGraphic" resourceName="platform:/plugin/Test/icons/full/obj16/Test.gif"/>
<colorGraphic32 xmi:type="utility:GIFFileGraphic" resourceName="platform:/plugin/Test/icons/full/obj30/Test.gif"/>
<composition>
<nodes xmi:type="ComIbmFileInput.msgnode:FCMComposite_1" xmi:id="FCMComposite_1_1" location="82,180" inputDirectory="c:\mbtest" filenamePattern="in.xml" processedFileAction="moveToArchive" messageDomainProperty="XMLNSC" recordDetection="parsedRecordSequence">
<translation xmi:type="utility:ConstantString" string="File Input"/>
</nodes>
<nodes xmi:type="ComIbmCompute.msgnode:FCMComposite_1" xmi:id="FCMComposite_1_2" location="327,173" computeExpression="esql://routine/#Test_Compute.Main">
<translation xmi:type="utility:ConstantString" string="Compute"/>
</nodes>
<nodes xmi:type="ComIbmFileOutput.msgnode:FCMComposite_1" xmi:id="FCMComposite_1_3" location="503,173" outputDirectory="c:\mbtestoutput" outputFilename="out.csv">
<translation xmi:type="utility:ConstantString" string="File Output"/>
</nodes>
<connections xmi:type="eflow:FCMConnection" xmi:id="FCMConnection_1" targetNode="FCMComposite_1_2" sourceNode="FCMComposite_1_1" sourceTerminalName="OutTerminal.out" targetTerminalName="InTerminal.in"/>
<connections xmi:type="eflow:FCMConnection" xmi:id="FCMConnection_2" targetNode="FCMComposite_1_3" sourceNode="FCMComposite_1_2" sourceTerminalName="OutTerminal.out" targetTerminalName="InTerminal.in"/>
<connections xmi:type="eflow:FCMConnection" xmi:id="FCMConnection_3" targetNode="FCMComposite_1_3" sourceNode="FCMComposite_1_1" sourceTerminalName="OutTerminal.EOD" targetTerminalName="InTerminal.EOD"/>
</composition>
<propertyOrganizer/>
<stickyBoard/>
</eClassifiers>
</ecore:EPackage>
|
test.esql
| Code: |
CREATE COMPUTE MODULE Test_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
-- CALL CopyMessageHeaders();
-- CALL CopyEntireMessage();
SET OutputRoot.Properties = InputRoot.Properties;
SET OutputRoot.Properties.MessageFormat = 'Text_CSV';
SET OutputRoot.Properties.MessageSet = 'M0P5VOS002001';
SET OutputRoot.Properties.MessageType = 'CVS_Message';
SET OutputRoot.MRM.Data.firstname = InputRoot.XMLNSC.person.firstname;
SET OutputRoot.MRM.Data.lastname = InputRoot.XMLNSC.person.lastname;
RETURN TRUE;
END;
CREATE PROCEDURE CopyMessageHeaders() BEGIN
DECLARE I INTEGER 1;
DECLARE J INTEGER;
SET J = CARDINALITY(InputRoot.*[]);
WHILE I < J DO
SET OutputRoot.*[I] = InputRoot.*[I];
SET I = I + 1;
END WHILE;
END;
CREATE PROCEDURE CopyEntireMessage() BEGIN
SET OutputRoot = InputRoot;
END;
END MODULE;
|
MessageSet Project:
| Code: |
.project
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>TestMsg</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>com.ibm.etools.mft.builder.xsi.MSetProjectBuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>com.ibm.etools.mft.flow.adapters.adapterbuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>com.ibm.etools.mft.flow.sca.scabuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>com.ibm.etools.msg.validation.mft.builder.msetbuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>com.ibm.etools.mft.bar.ext.barbuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>com.ibm.etools.msg.validation.msetnature</nature>
<nature>com.ibm.etools.mft.bar.ext.barnature</nature>
</natures>
</projectDescription>
|
csv.mxsd
| Code: |
<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:complexType name="CSV_Row_Type">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsStructRep dataElementSeparation="AllElementsDelimited" delimiter="," groupIndicator="" groupTerminator="" messageSetDefaultRep="Text_CSV"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:sequence>
<xsd:element fixed="FirstName" maxOccurs="unbounded" name="FirstName" nillable="true" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="9" messageSetDefaultRep="Text_CSV" precision="-1"/>
<tdsInclRep messageSetDefaultRep="Text_CSV" repeatingElementDelimiter=","/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:element fixed="LastName" name="LastName" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="8" messageSetDefaultRep="Text_CSV" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="CSV_Message_Type">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsStructRep dataElementSeparation="AllElementsDelimited" delimiter="<CR><LF>" groupIndicator="" groupTerminator="" messageSetDefaultRep="Text_CSV"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:sequence>
<xsd:element maxOccurs="unbounded" name="CSV_Row" nillable="false" type="CSV_Row_Type">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsInclRep messageSetDefaultRep="Text_CSV" repeatingElementDelimiter="<CR><LF>"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:element name="Data" type="Data_Type"/>
</xsd:sequence>
</xsd:complexType>
<xsd:element name="CSV_Message" type="CSV_Message_Type">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<MRMessage messageDefinition="/0/CSV_Message;XSDElementDeclaration$MRObject"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:complexType name="Data_Type">
<xsd:sequence>
<xsd:element name="firstname" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="9" messageSetDefaultRep="Text_CSV" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
<xsd:element name="lastname" type="xsd:string">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<tdsElemRep length="8" messageSetDefaultRep="Text_CSV" precision="-1"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:schema>
|
messageSet.mset
| Code: |
<?xml version="1.0" encoding="UTF-8"?>
<msgCoreModel:MRMessageSet xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:MSGModel="http://www.ibm.com/msgmodel/2003/MSGModel" xmlns:msgCoreModel="http://www.ibm.com/msgmodel/2003/MSGCoreModel" name="TestMsg" msetVersionNo="" defaultRep="//Text_CSV" currentMessageSetId="//@MRMessageSetID.0">
<MRMessageSetRep xsi:type="MSGModel:MRTDSMessageSetRep" name="Text_CSV" centuryWindow="53" timeZoneID="0" allowLenientDateTimes="false" messagingStandard="CSV" groupIndicator="" groupTerminator="" tagDataSeparator="" decimalPoint="." escapeCharacter="" reservedChars=",<CR><LF>"" outputCompressionTechnique="None" inputCompressionTechnique="None" defaultCCSID="367" booleanTrueRepresentation="1" booleanFalseRepresentation="0" booleanNullRepresentation="0" delimiter="," trimFixLengthString="NoTrim" deriveDefaultLengthFromLogicalType="true" quoteCharacter=""" deriveDefaultSignFromLogicalType="true">
<tagLength xsi:nil="true"/>
</MRMessageSetRep>
<MRMessageSetID repositoryId="-1067139129" messageSetId="1"/>
</msgCoreModel:MRMessageSet> |
[/b] |
|
| Back to top |
|
|
| diamondiona |
| Posted: Sat Jun 21, 2014 8:34 pm Post subject: |
|
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
I made progress. However, still didn't figure out.
Here is my test data:
<person>
<firstname>John</firstname>
<lastname>Haygen</lastname>
</person>
<person>
<firstname>Mike</firstname>
<lastname>Haygen</lastname>
</person>
Here is my message flow:
1. FileInputNode passed incoming message
2. computer node transform to CSV
SET OutputRoot.Properties = InputRoot.Properties;
SET OutputRoot.Properties.MessageFormat = 'Text_CSV';
SET OutputRoot.Properties.MessageSet = 'M0P5VOS002001';
SET OutputRoot.Properties.MessageType = 'person';
SET OutputRoot.MRM.Data.firstname = InputRoot.XMLNSC.person.firstname;
SET OutputRoot.MRM.Data.lastname = InputRoot.XMLNSC.person.lastname;
Here is my MessageSet definition:
PersonType
HeaderType
FirstName --fixed value 'FirstName' Occurrence 1..1
LastName -- fixed value 'LastName' Occurrence 1..1
DataType
firstname
lastname
My output is:
--blank row
John Haygen
--blank row
Mike Haygen
So my question is:
1 should the fixed value be used for output if element missed in ESQL. In this case, I didn't set header values.
2 How can I output the header value only at the first row of output file?
My project is blocking by this issue. Any input is truly appreciated!
Diamond |
|
| Back to top |
|
|
| kimbert |
| Posted: Sun Jun 22, 2014 1:40 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I am not sure that I understand what your problem is. Please post the output that you are getting and the output that you want.
And, please put them into two separate [code] sections. This is important - a [code] section makes the white space visible, and helps to avoid misunderstandings.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| diamondiona |
| Posted: Sun Jun 22, 2014 6:19 pm Post subject: |
|
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
Here is the output:
| Code: |
John, Haygen
Mike, Haygen
|
The problem is on the first and third rows, I got all empty values.
My goal is output like:
FirstName,LastName <--- Header Information
John,Haygen
Mike,Haygen |
|
| Back to top |
|
|
| diamondiona |
| Posted: Sun Jun 22, 2014 7:10 pm Post subject: |
|
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
Hi Kimbert,
Is it possible to achieve what I want? Please let me know if you need more information.
I appreciate for your help!
Diamond |
|
| Back to top |
|
|
| manoj5007 |
| Posted: Sun Jun 22, 2014 7:29 pm Post subject: |
|
|
Acolyte
Joined: 15 May 2013
Posts: 64
|
Hi diamondiona,
What you are trying to do is possible by making use of the group terminator and delimiters.
Create a group for firstname and lastname and terminate it using CRLF and create another group containing the details of the remaining message. |
|
| Back to top |
|
|
| diamondiona |
| Posted: Sun Jun 22, 2014 7:48 pm Post subject: |
|
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
I created two groups as you said. But no luck at all. Do you mind to create an example and send it to me through email? My email is: diamondiona@gmail.com.
Sorry for the rush because I need to get it done today.
Truly appreciate for everyone help! |
|
| Back to top |
|
|
| diamondiona |
| Posted: Sun Jun 22, 2014 8:01 pm Post subject: |
|
|
Acolyte
Joined: 19 Jun 2014
Posts: 50
|
| Also, I want to clarify that incoming message is XML with many records, parsed by FileInput Node. The message set is used for output CSV. |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Jun 23, 2014 12:22 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Your model structure is too complex. You do not need a separate structure for 'Header'. You simply need a Row that contains firstName and lastName.
You cannot use fixed values to output the columns headers. The MRM TDS parser will not use fixed/default values unless it is writing a fixed-length field ( i.e. Data Element Separation = Fixed Length ).
In your Compute node, you should add two lines of ESQL that create an extra first row for the column names. Look up the 'CREATE' statement with the FIRSTCHILD modifier. Or search this forum for examples.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
|
|
