| Author |
Message
|
| bdaoust |
 Posted: Mon Oct 26, 2015 6:00 pm Post subject: For Each / Else If - Map controlling sequence of output |
 |
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
Hello everyone.
I have an XML and I'm using a graphical map to do a For Each for a repeating
structure that looks like this:
| Code: |
<Colors>
<ShortDescription>Yellow</ShortDescription>
<Value>2</Value>
<ShortDescription>Blue</ShortDescription>
<Value>5</Value>
</Colors> |
So in the For Each, I have submap that does an IF ELSE IF.
Within the For IF ELSE IF, if the Shortdescription is Yellow, I then have a move transform to take value (2) to YellowCount in my DFDL
If it's blue, then move value to BlueCount in my DFDL.
In the XML Yellow can come before Blue or visa versa or not even be there at all. There are many different colors and each would have it's own Else If Conditional statement.
BTW what I'm working on is not colors, just trying to make my example simple.
My DFDL is a schema for comma delimited output with headers and column itegrity must be maintained. Meaning YellowCount always needs to be in the same column, BlueCount needs to be in the same columns, etc for each record.
Because in the XML the different colors don't show up in any order, my output tree can vary:
YellowCount
BluieCount
RedCount
or next record
BlueCount
YellowCount
RedCount
How can I force the output to place the values and maintain the same phystical strucution - meaning YellowCount always column1, BlueCount always column2?
In the Map I see order, but that seems to force an alphanumerical order. My output structure is defined by the user, so there is no logical order Like Blue,Red,Yellow (alphabetical) but yet it is Yellow,Blue,Red.
Thanks |
|
| Back to top |
|
 |
| bdaoust |
| Posted: Mon Oct 26, 2015 6:13 pm Post subject: |
|
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
To add:
In the map the group condition order looks right to me:
if ($ShortDecription = 'Yellow')
else if ($ShortDecription = 'Blue') |
|
| Back to top |
|
|
| timber |
| Posted: Tue Oct 27, 2015 2:30 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
In your DFDL xsd, are 'BlueCount' and 'YellowCount' using the same element name( with maxOccurs > 1 ) or different names?
Because if they are using different element names then
a) the mapping node should always write them in schema declaration order
and
b) DFDL should ensure that they are written to the bitstream in the correct order.
There is something unusual going on here. Are you sure that your example scenario is exactly equivalent to what is going on in the real message flow? |
|
| Back to top |
|
|
| bdaoust |
| Posted: Tue Oct 27, 2015 3:31 am Post subject: |
|
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
I actually tried it both ways. Meaning the DFDL having it's own element per collor and in the map I had the appropriate IF/ELSEIF statement wired to the color that matched that condition. The second way I tried is just one element called Color with Max Occurs >1
It's almost like the order of the output is being determined by the order in which the colors appears in the For Each loops while doing through the array input element.
My flow is pretty simple - Inpiut Node - Map Node with Inpuit Source set as the XML and Output Traget set as the DFDL I created. The DFDL I just created via the wizard and added the fields I needed, Yellow, Blue, Red, etc.
I even recreated the map thinking maybe I set some property wrong or something.
But like you suggested, the DFDL should dictate the physical output structure and it's not. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Oct 27, 2015 3:55 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| It's almost like the order of the output is being determined by the order in which the colors appears in the For Each loops while doing through the array input element. |
The phrase 'its almost like' is a red flag. You don't need to speculate about this. Insert a Trace node in the message flow and look at the message tree that is being built by the Mapping node. Then you will know for certain.
| Quote: |
| the DFDL should dictate the physical output structure and it's not. |
It looks like that from where you are standing. You need to shift your position from 'its not doing what I expected' to 'lets find out what it is doing, and why'.
If you cannot make progress, please post all of the following:
- your output message tree structure ( using a Trace node after the Mapping node )
- the DFDL XSD
- the output
Please use code tags for all of the above - it makes everything so much more readable for those of us who are trying to assist you.
Feel free to use the toy example instead of the real data if you are concerned about data privacy. |
|
| Back to top |
|
|
| bdaoust |
| Posted: Tue Oct 27, 2015 5:56 am Post subject: |
|
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
My broker doesn't run on a machine that I can access - so I need admins involved and depending on what they have going on - it can take a while.
I do have access to a text file on the server - so I specify file trace and the location of the file.
What pattern can I use to get what a user trace would give me?
Thanks |
|
| Back to top |
|
|
| bdaoust |
| Posted: Tue Oct 27, 2015 6:26 am Post subject: |
|
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
By the way I know I can do ${Root}.
However, if I wanted the same user trace output without needing to run any commands (since I don't have access to the server), can it be done? Again, I do a a file I can retrieve to the server, but once I choose File instead of User Trace, I'm no longer doing the same type of trace I would assume. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Oct 27, 2015 6:34 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The Trace node outputs the same data regardless of destination.
If you were really feeling clever, you might be able to get something usable with a FileInput node.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| bdaoust |
| Posted: Tue Oct 27, 2015 6:41 am Post subject: |
|
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
I guess I mean, I've seen a user trace that gives much more verbose of details, like when it's executing a map - it gives detail of the logic that it's executing and what not.
The trace of the message flow and what it's doing rather than just tracing the output of a node. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Oct 27, 2015 7:08 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You have to run commands on the broker server in order to process usertrace into text files.
But you were asked for Trace node output to resolve your problem, since it appears to be with the structure of your message, rather than with the execution of your map node.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| bdaoust |
| Posted: Tue Oct 27, 2015 7:11 am Post subject: |
|
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
Yup - getting that now. 
Thanks for clarifying out my thought on user trace and needing to process usertrace into text files. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Oct 27, 2015 7:34 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| My broker doesn't run on a machine that I can access |
Download the developer edition of IIB for free, and create/test your DFDL model and map on your own box. Much faster and easier than trying to develop everything remotely on a locked-down machine. |
|
| Back to top |
|
|
| bdaoust |
| Posted: Tue Oct 27, 2015 9:14 am Post subject: |
|
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
Admins were looking at me getting a local broker  Thank god.
Anyways: here is the output after the map:
| Code: |
( ['MQROOT' : 0x116d0f830]
(0x01000000:Name):Properties = ( ['MQPROPERTYPARSER' : 0x11866bd30]
(0x03000000:NameValue):MessageSet = '' (CHARACTER)
(0x03000000:NameValue):MessageType = '{}:HeaderForPremiumValidation' (CHARACTER)
(0x03000000:NameValue):MessageFormat = '' (CHARACTER)
(0x03000000:NameValue):Encoding = 546 (INTEGER)
(0x03000000:NameValue):CodedCharSetId = 437 (INTEGER)
(0x03000000:NameValue):Transactional = TRUE (BOOLEAN)
(0x03000000:NameValue):Persistence = FALSE (BOOLEAN)
(0x03000000:NameValue):CreationTime = GMTTIMESTAMP '2015-10-27 16:58:59.620' (GMTTIMESTAMP)
(0x03000000:NameValue):ExpirationTime = -1 (INTEGER)
(0x03000000:NameValue):Priority = 0 (INTEGER)
(0x03000000:NameValue):ReplyIdentifier = X'000000000000000000000000000000000000000000000000' (BLOB)
(0x03000000:NameValue):ReplyProtocol = 'MQ' (CHARACTER)
(0x03000000:NameValue):Topic = NULL
(0x03000000:NameValue):ContentType = '' (CHARACTER)
(0x03000000:NameValue):IdentitySourceType = '' (CHARACTER)
(0x03000000:NameValue):IdentitySourceToken = '' (CHARACTER)
(0x03000000:NameValue):IdentitySourcePassword = '' (CHARACTER)
(0x03000000:NameValue):IdentitySourceIssuedBy = '' (CHARACTER)
(0x03000000:NameValue):IdentityMappedType = '' (CHARACTER)
(0x03000000:NameValue):IdentityMappedToken = '' (CHARACTER)
(0x03000000:NameValue):IdentityMappedPassword = '' (CHARACTER)
(0x03000000:NameValue):IdentityMappedIssuedBy = '' (CHARACTER)
)
(0x01000000:Name):MQMD = ( ['MQHMD' : 0x1171107d0]
(0x03000000:NameValue):SourceQueue = 'TmpHold' (CHARACTER)
(0x03000000:NameValue):Transactional = TRUE (BOOLEAN)
(0x03000000:NameValue):Encoding = 546 (INTEGER)
(0x03000000:NameValue):CodedCharSetId = 437 (INTEGER)
(0x03000000:NameValue):Format = 'MQSTR ' (CHARACTER)
(0x03000000:NameValue):Version = 2 (INTEGER)
(0x03000000:NameValue):Report = 0 (INTEGER)
(0x03000000:NameValue):MsgType = 8 (INTEGER)
(0x03000000:NameValue):Expiry = -1 (INTEGER)
(0x03000000:NameValue):Feedback = 0 (INTEGER)
(0x03000000:NameValue):Priority = 0 (INTEGER)
(0x03000000:NameValue):Persistence = 0 (INTEGER)
(0x03000000:NameValue):MsgId = X'414d512044494942392020202020202055ed58852e96b00a' (BLOB)
(0x03000000:NameValue):CorrelId = X'000000000000000000000000000000000000000000000000' (BLOB)
(0x03000000:NameValue):BackoutCount = 0 (INTEGER)
(0x03000000:NameValue):ReplyToQ = ' ' (CHARACTER)
(0x03000000:NameValue):ReplyToQMgr = 'DIIB9 ' (CHARACTER)
(0x03000000:NameValue):UserIdentifier = 'mqm ' (CHARACTER)
(0x03000000:NameValue):AccountingToken = X'16010515000000755df67e8640542f64618937bb67010000000000000000000b' (BLOB)
(0x03000000:NameValue):ApplIdentityData = ' ' (CHARACTER)
(0x03000000:NameValue):PutApplType = 11 (INTEGER)
(0x03000000:NameValue):PutApplName = 'q utils\rfhutil\rfhutilc.exe' (CHARACTER)
(0x03000000:NameValue):PutDate = DATE '2015-10-27' (DATE)
(0x03000000:NameValue):PutTime = GMTTIME '16:58:59.620' (GMTTIME)
(0x03000000:NameValue):ApplOriginData = ' ' (CHARACTER)
(0x03000000:NameValue):GroupId = X'000000000000000000000000000000000000000000000000' (BLOB)
(0x03000000:NameValue):MsgSeqNumber = 1 (INTEGER)
(0x03000000:NameValue):Offset = 0 (INTEGER)
(0x03000000:NameValue):MsgFlags = 0 (INTEGER)
(0x03000000:NameValue):OriginalLength = -1 (INTEGER)
)
(0x01000000:Name):DFDL = ( ['dfdl' : 0x116edafd0]
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation = (
(0x03000000:NameValue):COLARiderAnnualPremium = 'COLA3' (CHARACTER)
)
(0x01000000:Name):HeaderForPremiumValidation = (
(0x03000000:NameValue):FIORiderAnnualPremium = 'FIO' (CHARACTER)
)
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation =
(0x01000000:Name):HeaderForPremiumValidation =
)
)
|
And the DFDL Schema:
| Code: |
<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:csv="http://www.ibm.com/dfdl/CommaSeparatedFormat" xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:ibmDfdlExtn="http://www.ibm.com/dfdl/extensions" xmlns:ibmSchExtn="http://www.ibm.com/schema/extensions">
<xsd:import namespace="http://www.ibm.com/dfdl/CommaSeparatedFormat" schemaLocation="IBMdefined/CommaSeparatedFormat.xsd"/>
<xsd:annotation>
<xsd:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:format documentFinalTerminatorCanBeMissing="yes" encoding="{$dfdl:encoding}" escapeSchemeRef="csv:CSVEscapeScheme" ref="csv:CommaSeparatedFormat"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:element ibmSchExtn:docRoot="true" name="HeaderForPremiumValidation">
<xsd:complexType>
<xsd:sequence>
<xsd:element minOccurs="0" name="FIORiderAnnualPremium" type="xsd:string"/>
<xsd:element minOccurs="0" name="COLARiderAnnualPremium" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema> |
And the output: (using RFHUTIL to display)
00000000 ,COLA3FIO
I know that the delimiter is not set I striped down my example to make it easy.
So the desired output results is :FIO,COLA3
In other testing, I did try to set the sequence to 'order', but that didn't help. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Oct 27, 2015 9:45 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
If you're doing a for loop on the input tree, I believe that the map node will build the output tree in the order received.
In your case, you are mapping to a repeating structure, where you need specific instances of the repeat in order, rather than different elements in order. So Order doesn't really help.
You'll need to do something to figure out where to assign a given field in the message tree in your for loop. using the [n] or something. The difficulty comes in if the [n] has to be variable (i.e. if "COLA3" is the first non-empty field in the input, then it has to be the first field in the output). This could maybe be done with the loop counter. But you'd have to do your IF expressions in the right order from top to bottom.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| bdaoust |
| Posted: Tue Oct 27, 2015 10:02 am Post subject: |
|
|
Centurion
Joined: 23 Sep 2010
Posts: 130
|
Yes, I'm doing a For Loop in the input
Then within the submap doing the IF/ELSE IF (in order BTW - so FIO first, COLA second, etc) and then a MOVE to the output element.
I was hoping the DFDL schema would enforce the output regardless of the output tree order. Guess that is not the case?
| Quote: |
| In your case, you are mapping to a repeating structure |
Not sure what you mean here, my output doesn't need to be repeating - the output only has one element for each match in For/Each. Then I set output elements to a null value if there isn't a match. Guess that concept won't work?
I did get it to 'work' by having a separate For Each for each output element I wanted to create, but I thought this is probably a really bad design.
| Quote: |
| The difficulty comes in if the [n] has to be variable (i.e. if "COLA3" is the first non-empty field in the input, then it has to be the first field in the output). |
This is okay. So if FIO input is blank then a blank is fine - meaning a valid output can look like:
FIO,,SIS
So in between FIO and SIS should be COLA but in this case the input did not have COLA. That is fine and expected at times. As long as order of the elements remain constant because the user will expect FIO,COLA and SIS always to be in the same column.
| Quote: |
| This could maybe be done with the loop counter. |
Are you meaning in ESQL? I'm very tempted to take this to ESQL but determine to get it working in a mapping node. |
|
| Back to top |
|
|
|
|
