| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Message set using delimiters count. |
« View previous topic :: View next topic » |
| Author |
Message
|
| mapa |
 Posted: Thu Apr 19, 2012 10:36 am Post subject: |
 |
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
| Broker wrote: |
I specified it's Properties as
Third payment:DataElement seperation: All Elements Delimited
Delimiter: |<CR><LF>
For all sequence: DataElement separation: Use Data Pattern
For Header, Body, trailer: All Elements Delimited
And
Data patterns as u specified as:
Header: .{1}|.{8}|.{6}|
Body: {12}|.{12}|.{1}|.{6}|.{16}|.{16}|.{8}|.{8}|.{8}|.{8}|.{13}|.{9}|.{4}|.{5}|.{13}.{9}|.{13}|
trailer:.{5}|.{13}|.{13}|
|
Well, the broker has no way of knowing when the Body sequence ends and the trailer sequence starts as far as I can tell.
I know that it is not always possible to get a change to the inbound format, but it is well worth a shot in your case.
An alternative is if you can get them to extend the header and in it write how many bodies and how many trailer lines it contains. Then you can use that as a "Repeat Reference". |
|
| Back to top |
|
 |
| kimbert |
| Posted: Thu Apr 19, 2012 10:53 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| Well, the broker has no way of knowing when the Body sequence ends and the trailer sequence starts as far as I can tell. |
It does - that's what the Data Pattern is for.
| Quote: |
| I know that it is not always possible to get a change to the inbound format, but it is well worth a shot in your case. |
There is no need to change the format - the TDS parser is perfectly capable of parsing this. I know, because I have helped several others with similar formats.
@Broker: Please take a debug-level user trace, as requested before. Without that, you are very unlikely to find out where the problem is.
Oh, and next time you post, see whether you can get those code tags working  |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 19, 2012 10:55 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| kimbert wrote: |
| Oh, and next time you post, see whether you can get those code tags working |
I'd review the check boxes you have selected on the posting page. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mapa |
| Posted: Thu Apr 19, 2012 11:08 am Post subject: |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
| kimbert wrote: |
| Quote: |
| Well, the broker has no way of knowing when the Body sequence ends and the trailer sequence starts as far as I can tell. |
It does - that's what the Data Pattern is for.
| Quote: |
| I know that it is not always possible to get a change to the inbound format, but it is well worth a shot in your case. |
There is no need to change the format - the TDS parser is perfectly capable of parsing this. I know, because I have helped several others with similar formats.
@Broker: Please take a debug-level user trace, as requested before. Without that, you are very unlikely to find out where the problem is.
Oh, and next time you post, see whether you can get those code tags working |
Oh, my bad then.
Interesting, I've been fortunate then I guess. I've made several hundreds of message sets over the years but never had to use Data Pattern.
Will have to abort tonights Java prototyping with reading up on Data Pattern and experiment a little  |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 19, 2012 11:21 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mapa wrote: |
| Interesting, I've been fortunate then I guess. I've made several hundreds of message sets over the years but never had to use Data Pattern. |
It's very seldom that you do need to resort to a Data Pattern; most file formats can be modeled without them.
I choose my words carefully; Data Pattern should be a last resort when all other methods have failed. It's the most expensive of the parsers.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mapa |
| Posted: Thu Apr 19, 2012 11:26 am Post subject: |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
Yes, that was my understanding as well.
I only though you used it for parsing records.
Not that you could use it to actually distinguish between two repeating records with different lengths/number of field delimiters that are terminated by LF like in this case. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Apr 19, 2012 1:28 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| I've made several hundreds of message sets over the years but never had to use Data Pattern. |
In that case, you've been fortunate - I know of dozens of formats that TDS could never handle without using Data Patterns. Or maybe you're very good at getting the sender to change their format - whatever works for you
| Quote: |
| Data Pattern should be a last resort when all other methods have failed. It's the most expensive of the parsers. |
That's true - if you can do without data patterns then there's no point in paying the CPU cost.
However, I think it's time to balance this up a little. Since v6 data patterns have become a little more efficient ( along with the rest of the TDS parser ). In any case, if there is no other way to do the job, there is no shame in creating a message set that contains data patterns.
| Quote: |
I only though you used it for parsing records.
Not that you could use it to actually distinguish between two repeating records with different lengths/number of field delimiters that are terminated by LF like in this case. |
Here's how it works: if the Data Element Separation is set to Use Data Pattern, then each member of that complex type ( or group ) must have its 'Data Pattern' property set to a valid regular expression.
When the parser arrives at each member of the type/group it executes the data pattern against the input message and notes how many characters it matched. That number is now the length for the entire element/group.
IF the matched length is zero characters ( no match ) AND the element is optional or is a child of a choice THEN the parser does not attempt to parse it at all.
If the group member is a simple element, the matched length is used as the length of that string/integer/whatever.
If the group member is a complex element, the matched length defines a fixed-length 'window' in the input message. All the children of the complex element must fit into that 'window'
If the group member is a group ( i.e. not an element, but an XML Schema sequence or choice ) then it works the same way as for a complex element - the matched length defines a fixed-length 'window' for the entire group.
So for this CSV example, Broker is correct in defining a Data Pattern that matches the entire body record. If it only matched the first 10 fields, the 'window' for the body would only encompass the first 10 fields and a parsing error would result.
So what's the problem with Broker's message set? Impossible to say without seeing a trace. My best guess is that min/maxOccurs have not been set correctly, but that's just a shot in the dark. |
|
| Back to top |
|
|
| mqsiuser |
| Posted: Thu Apr 19, 2012 2:09 pm Post subject: |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
Thank you for the explanations, that is really interesting and certainly helps.
And yes... for finding out message set issues a trace can really help (a consultant once told me ... he created the first msg sets for us/for me ).
@OP: On the "Logical Properties > Local Element" you need to properly set the occurrences: "Min Occurs=1" (or 0), "Max Occurs=-1" (-1 is infinite... or you set any other finite number (ofc)) for every group and for every field ! ... there also is the "Name" and for the fields (simple types) on the "Type" probably just use "string" (and use CASTs in ESQL... which will give you a better / different way to deal with type errors).
Try to include the line ending in your reg exp "\n" (or "\r\n"! Probably don't try around, but find out (about your bits/bytes (of carriage return and line feed) in your msg and by looking into code page tables)) !... I had always included "\r?\n", since that would work for Windows and Unix/Linux files .
Use reg exps on the group / complex types (CT) and not necessarily on the fields (eg. just specify "Length" on the simple types / field level) (or/but you can also work with reg exp on field level):
Your reg exps (on the group/CT) must exactly match all the characters/bytes that are there (RFH-Util can show what really is there in terms of bytes). The (group) reg exp must "consume" the full message (except: e.g. the very last \r\n is missing... that might/will still work... Note that this is the case in files usually (on the last line))... the trick with the reg exps (on group level) is: The parser uses it to uniquely identify the "right" group... so probably include some unique characters which occur only in that group / CT.
Probably use the "Repeating Element Delimiter" (in the msg-set-def: "|") for the field-level, so that the fields of your input-msg do not contain these "control" characters. You need to set this delimiter "|" (additionally to having it in the reg exps of the group) on the field level !
So your fields will just be like "Length=5" and (probably) "repeating element delimiter = '|'" (and set "Min Occurs=1 (or 0)", "Max Occurs=1", "Padding Character" and "Justification"), while your group must be reg exps like ".{5}|.{5}|.{5}|\r?\n" (or just ".{18}\r?\n" ! )
Note that in the reg exp it is "\r\n" while in the "Repeating Element Delimiter" field it would be "<CR><LF>" !... though I think you don't need to specify it in your case (the reg exp does it on that level).
The thing that helped me best on msg set creation/issues were: Some good samples (as I said: an external consultant created these for us). Once we/I had them I could look there and create msg sets on my own ... the thing/problem with error traces/msgs/info is: They do not (necessarily) tell (a beginner) the correct solution. So some heads up knowledge first is desirable/necessary... e.g. examples.
Since you are not there (yet), another tipp is: Try to parse something easy/easier first, e.g. Header-lines only (with reg exp ofc !). Once that works go on with header + lines, once that works ... add the trailer(s) to your msg (and msg set).
Works for me, works on my laptop
There are different ways of approaching a problem. Look at the ones available and pick the one(s) that work(s) for you.
_________________
Just use REFERENCEs |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
