| Author |
Message
|
| ak |
 Posted: Tue Jun 05, 2007 12:48 am Post subject: How can I define a message set that is both TDS and CWF |
 |
|
Apprentice
Joined: 07 Aug 2006
Posts: 48
Location: England
|
I need to create an MRM but the input message seems to be both TDS and CWF! Has anyone encountered this before ?
Here is an example of a single input message
Record1003004005006007008
Record2004007008009001006
Record3004007008009001006
1. Each record can clearly and easily be defined as CWF.
2. The biggest problem is that all the 3 records make up a single MQ message and are delimited by a line feed charater and hence TDS format.
Any help will highly appreciated.
ak |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Tue Jun 05, 2007 2:16 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You're overthinking this problem.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| ak |
| Posted: Tue Jun 05, 2007 2:29 am Post subject: |
|
|
Apprentice
Joined: 07 Aug 2006
Posts: 48
Location: England
|
Oh that is helpful! This is a genuine problem a legency system sends single messages comprised of well defined CWF records as above. Each record is about 420 charaters long and records are delimited using a <CR><LF>. Is that TDS again?
I guess my question was if anyone is encountered this problem before. if not I will resort to a Java Compute node that can buffer all the records, parse them and probably output one XML message with the require values, but this is the last resort. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jun 05, 2007 2:42 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The records are fixed in length, yes?
Always?
Then you don't need TDS.
You would only need TDS if each record was of variable length. And maybe not even then.
On the other hand, you don't need CWF at all. You've got a "tag", and a delimiter. And TDS is entirely capable of parsing fixed length data, with no delimiters or tags - data element separation "Tagged, Fixed Length" - or you could use VLED and have no variable length elements.
Either way, you don't need BOTH CWF and TDS.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jun 05, 2007 3:23 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Jeff's right - assuming that your input messages do not contain any binary data ( packed decimals, binary integers etc ). |
|
| Back to top |
|
|
| ak |
| Posted: Tue Jun 05, 2007 7:14 am Post subject: |
|
|
Apprentice
Joined: 07 Aug 2006
Posts: 48
Location: England
|
Jeff is right because he makes the assumption that records have variable lenght butthey not. Infact each record can be parsed very well as CWF. Unfortunately I have to group a variable number of records together to make a single Mq message and hence the need for record delimeters - which then sounds to me like TDS physical format?
Sounds like a no go for everybody, so I will try plan B.
thanks everybody!! |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jun 05, 2007 7:32 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Hi ak,
- Do your input messages contain any CWF-only types ( like packed decimals)?
- Are the records different, or is it just a repeating record with a variable-length field as its final member?
- If the records are different, is there a tag identifying each record?
- Why do you need a delimiter to separate the records? Please be specific about this. If the last field in each record is variable-length, could you perhaps use a null terminator instead of <CR><LF>? If so, you could use CWF physical type 'Null terminated string' |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jun 05, 2007 7:36 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You still haven't shown that you need to go to a custom parsing option, or TDS instead of CWF.
I'm making no assumptions about the data, I'm using only the information you have presented.
In general, CWF can't handle an unspecified number of repeats. Since you didn't mention that there was an unspecified number of repeats, there was no reason to rule out CWF. If you're saying that there IS an unknown number of repeats, then you do need to rule out using CWF.
You can, however, just use TDS - as I said, you can model fixed length records in TDS, without bothering with tags or delimiters.
HOWEVER, as Kimbert said - TDS can't handle binary data. Nothing in the sample data you showed appears to be binary data. But maybe you've forgotten to explain that your message does contain binary data - like you forgot to explain that it had an unknown number of repeats.
If this is the case, that your message has both binary data and an unknown number of repeats... I still have a solution for you that does not require that you do a lot of work.
Model the message as CWF. Add a single integer field to the model, that is the repeat count, and appears as the first element. Read the message as BLOB, get the length of the BLOB data and use it to calculate your repeat count. Prepend this length count to the BLOB data, and parse the message as CWF. If you need to output a message in this manner, then set the repeat count to the correct number, use ASBITSTREAM to serialize the message tree to a BLOB field and then remove the repeat count.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| ak |
| Posted: Tue Jun 05, 2007 8:44 am Post subject: |
|
|
Apprentice
Joined: 07 Aug 2006
Posts: 48
Location: England
|
Jeff/Kimbert
Sorry if I missed out any information, I will instead rephrase the problem.
1. I have a properly defined spec for all the types of records and I have modelled MRM as CWF for each. None of them has repeats and the elements are fixed length strings which made that very straight forward.
2. I then realised that a meaningful bussiness message or MQ Message will be composed of a variable number of those records separated or delimited by a line feed. Information required to process the MQ message further is held with this batch of records, so I need to receive them at the same time. Since WMB is stateless I cannot even think of receiving record by record and buffer them.
It is the fact that a single MQ message is a batch of these well defined records(fixed length strings) delimited by a line feed that threw me away.
Probably this makes sense now. If the records held a single string value TDS would have been ideal but unfortunately some records hold upto 30 fixed length elements
Ta
ak |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jun 05, 2007 9:22 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The fact that a single message is built up of multiple records has nothing to do with using TDS or CWF.
The fact that a single record is made up of one or more than one field has noting to do with using TDS or CWF.
CWF can easily process this data, using your existing models, IF AND ONLY IF there is something that tells it HOW MANY records exist. This could be, for example, in a header record.
TDS can easily process this data, using your existing models, assuming you set the equivalent and needed properties on the TDS physical layer. AND assuming that your data has no binary fields in it.
You can include the linefeed in your model as a fixed length field at the end of your record if you use CWF.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jun 05, 2007 3:31 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
OK - the issues are a little clearer now. Here's what I understand. Read this carefully, line by line, and correct anything that is wrong:
1. Your message consists of lines of text
2. Lines are separated by <CR><LF>
3. Each line contains a different record type
4. The record type is identified by the first fixed-length field on each line
| Quote: |
| Information required to process the MQ message further is held with this batch of records |
What information. Is it in one of the records, or in a header record, or what?
| Quote: |
| If the records held a single string value TDS would have been ideal but unfortunately some records hold upto 30 fixed length elements |
That's just plain wrong - TDS can handle almost any text format. |
|
| Back to top |
|
|
| ak |
| Posted: Wed Jun 06, 2007 12:56 am Post subject: |
|
|
Apprentice
Joined: 07 Aug 2006
Posts: 48
Location: England
|
Kimbert
Your analysis is right and to clarify on the Quotes.
By information I meant, each record type consists of fixed length predifined values, and some records have more than 30 fields so I really need to use the MRM parser (CWF) to access them.
The problem is that an MQ messages have varying number of records batched together. The only obvious thing is that they are separated/delimited by a line feed.
If this helps at all, my current workaround is to read the MQ Message in as blob, using ESQL strip the blob into records using the line feed character as a delimeter. After which I SUBSTRING each record to extract the information or values required to build up this XML message that I propagate out. I have also tried a Java Compute node using the same idea. The down side to this is that the output is an XML message, I prefer an MRM as I need to use the Mapping Node to map the message to different other MRM messages.
Ta. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jun 06, 2007 1:43 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You can do everything you need to do using TDS.
UNLESS your records have binary fields in it.
1) Add a TDS physical layer to your CWF message set
2) Create a new Complex Type. This should have a type composition of "choice", and contain elements of the types that describe each of your records - So a Record1 element, Record2, etc.
3) Set the correct TDS properties your message. The new type should be Tagged, Fixed Length, with a group delimiter of your line feed. Each Record type should be Fixed Length. Each Record Element in the choice type should have the record code set as the Tag.
4) Create a new Message, that consists of an element of the choice type from 2, that can repeat an unlimited number of times.
5) debug
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| ak |
| Posted: Wed Jun 06, 2007 1:59 am Post subject: |
|
|
Apprentice
Joined: 07 Aug 2006
Posts: 48
Location: England
|
Thanks,
We seem to be speaking thesame languange now CWF working together with TDS. I will try this and let you know of the outcome, I always believed there was a way out but so scared to try incase I waste alot of time and lose faith in the tool.
ak |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jun 06, 2007 2:03 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
No, we're not.
You don't need CWF, and can't use it because of the unknown number of repeats and because of the fact that you've got a collection of records where each record type is a different length.
This is NOT using BOTH CWF and TDS. This is just using TDS.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
