| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| [UNSOLVED]problem with TDS modelling a CSV |
« View previous topic :: View next topic » |
| Author |
Message
|
| jefflowrey |
 Posted: Mon May 19, 2003 6:02 am Post subject: [UNSOLVED]problem with TDS modelling a CSV |
 |
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I'm trying to model a simple CSV file. For my test data, and probably for most real production data, the very first field is optional and very likely not populated. However, it's not possible to change the output layout to drop the field.
So, the records in the file are always going to start with just a ','.
The difficulty I'm having is that the TDS parser appears to be SKIPPING that delimiter, and taking the first value AFTER it as the first field.
That is, if I have a record like ",abc,123,abc", and four fields defined in the model , it's taking 'abc' as the value of the first field and not as the value of the SECOND field.
What am I doing wrong?
Last edited by jefflowrey on Mon May 19, 2003 12:50 pm; edited 3 times in total |
|
| Back to top |
|
 |
| fcotait |
| Posted: Mon May 19, 2003 6:40 am Post subject: |
|
|
Acolyte
Joined: 28 Feb 2002
Posts: 63
Location: Sao Paulo - Brazil
|
To do this, you need to create a Compound Type with the optionals fields and a Compound Type with the requeried fields
For more details, see Manual "Working with Messages", chapter 7, page 152 "Omission and truncation of elements"
_________________
Filipe Cotait
IBM Certified System Administrator - WebSphere MQ
IBM Certified Specialist - MQSeries, WebSphere MQ Integrator |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon May 19, 2003 7:28 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| I discovered the issue. I had the delimiter on my message compound type set to ',' as well as the delimiter on my data compound types. So that's why it was swallowing the first ',' in the record. I changed it to '<CR><LF>', and it's working now. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon May 19, 2003 1:00 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Actually, I haven't completely solved my problem. I got it to properly identify that the FIRST field was optional and missing.
Now I'm having trouble with when the LAST field is optional and missing.
My message model is set up like this:
- Message
- HeaderRecord
- HeaderField1
- HeaderField2
- ...
- HeaderFieldN
- BodyRecord
- BodyField1
- BodyField2]
- ...
- BodyFieldN
I have the Delimiter for the Message set to "<CR><LF>". I have the delimiters set for HeaderRecord and BodyRecord set to ",". All compound types are set to 'All fields delimited'. The HeaderRecord is set to not repeat. The BodyRecord is set to {Repeat yes, min 0, max 99999999, repeating deliminter <CR><LF>}
All fields in my messages are represented with a delimiter. When any one of my records ENDS with ",<CR><LF>", representing a missing last field (which is set to {repeat yes, min 0, max 1, repeat deliminter ","}) then the parsing of the message terminates at that point, and I get an exception that all data in the bitstream was not used for processing the message.
I have tried changing the repeat delimiter on the last body field to something that does not exist in the message, and I get the same problems. |
|
| Back to top |
|
|
| fcotait |
| Posted: Tue May 20, 2003 3:54 am Post subject: |
|
|
Acolyte
Joined: 28 Feb 2002
Posts: 63
Location: Sao Paulo - Brazil
|
Every fields can be optional, but the last one.
I got the same error and the solution is exactaly as written in the manual I sent you last message.
Good Lock
_________________
Filipe Cotait
IBM Certified System Administrator - WebSphere MQ
IBM Certified Specialist - MQSeries, WebSphere MQ Integrator |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 20, 2003 6:15 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I did actually read that section of the manual. However, I believe that I'm already doing what it suggests.
To quote
| Quote: |
| any elements that aremissing from the end of the compound type must be indicated by the application that creates the message in one of two ways. |
As I stated, the missing element is indicated in the message by a trailing delimeter for my child compound type.
The section goes on to say
| Quote: |
| If you have set the Delimiter property for the compound type to a value that does not match the value that you have set for the Delimiter property for any of the compound type’s parent types, the elements at the end of the message can be indicated by the occurrence of a Delimiter of one of its parents after the last actual element in the compound type data. |
In fact, this is the case. The delimiter for my compound type is ',' and the delimiter for it's only parent is '<CR><LF>'.
Are you saying that I need to subdivide my BodyRecord into two compound types - with one type holding only the last record? So then the message would look like
- Message
- HeaderRecord
- HeaderField1
- HeaderField2
- ...
- HeaderFieldN
- BodyRecord
- BodyMainRecords
- BodyField1
- BodyField2
- ...
- BodyFieldN-1
- BodyOptionalRecord
Wouldn't that just add to the confusion?
Or are you suggesting I modify the data in the file? |
|
| Back to top |
|
|
| fcotait |
| Posted: Tue May 20, 2003 8:02 am Post subject: |
|
|
Acolyte
Joined: 28 Feb 2002
Posts: 63
Location: Sao Paulo - Brazil
|
In my case, I had to modify the data in the file for it works
The important is the last field have a value, or the next value will be the Carriage Return (<CR><LF>) and the map will fail.
Try this with example:
| Code: |
MESSAGE_CSV
+ LOOP (Compound Type)
+ OPTIONAL_FIELDS (Compound Type)
- Field1 (ELEMENT)
- Field2 (ELEMENT)
- Field3 (ELEMENT)
- Field4 (ELEMENT)
+ FINALIZE (ELEMENT) |
Compound Type MESSAGE_CSV and OPTIONAL_FIELDS
Type Composition: Sequence
Data Element Separation: Variable Length Elements Delimited
Delimiter: ,
Compound Type LOOP
Type Composition: Sequence
Data Element Separation: Variable Length Elements Delimited
Delimiter: ,
Connection: Yes
Min: 1
Max: 999999
Repeating Element Delimiter: <CR><LF>
Don´t put Delimiter for the Message set.
File Data Example
1,123,12345,1234567,X
1,,12345,1234567,X
1,123,,1234567,X
1,123,12345,1234567,X
1,123,,,X
if you don´t want to use the Carriage with the Repeating, you have to use a TAG.
_________________
Filipe Cotait
IBM Certified System Administrator - WebSphere MQ
IBM Certified Specialist - MQSeries, WebSphere MQ Integrator |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 20, 2003 9:28 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
That means I'm going to have to read the message as a BLOB, parse it to either add a final field, or otherwise ensure that the last field is not missing.
I can't change the application producing the data, as it's an external application. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
