ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportMixed data inpu, ASCII and EBCDIC hex data parsing with DFDL

Post new topicReply to topic
Mixed data inpu, ASCII and EBCDIC hex data parsing with DFDL View previous topic :: View next topic
Author Message
mindspace
PostPosted: Tue May 01, 2018 11:54 am Post subject: Mixed data inpu, ASCII and EBCDIC hex data parsing with DFDL Reply with quote

Apprentice

Joined: 22 Jan 2013
Posts: 26

Hi,

I am struggling on how to handle below situation..

Input: Message format similar to ISO8583-87, with additional header elements(ISO VISA NET-BASE I Technical Specifications)

Data entering into IIB:
160101005800000085640400000000000000000000090302C22000000800000000800020080000021045495160977726310426191349005255F8F1F1F6F1F9F0F0F5F2F5F5220400F102C5F20BF4F3F04040404040404040

Breakup of above message:
Header - 160101005800000085640400000000000000000000090302
Bitmaps - C2200000080000000080002008000002
Data -
ActNumber_Field2-104146516097772631(prefix 10 means 16 digits)
Date_field7-0426191349
Trace Number -005255
Reference number_field 37- F8F1F1F6F1F9F0F0F5F2F5F5
Rest of the data - 220400F102C5F20BF4F3F04040404040404040

This data is received over TCP input node, this data has mixed format, some of them are in string format(Act number, date and trace number) and some of the data is in Hex format(Reference number- ebcdic,fixed length)

I am using the standard DFDL model provided by IBM for ISO8583-87 and trying to make changes to suit my needs.

I have set TCP input node to BLOB, and then in compute node trying to modify the data...before sending to RCD node with DFDL.

So the BLOB content after TCP node is

31363031303130303538303030303030383536343034303030303030303030303030303030303030303030393033303243323230303030303038303030303030303038303030323030383030303030323130343534393531363039373737323633313034323631393133343930303532353546384631463146364631463946304630463546324635463532323034303046313032433546323042463446334630343034303430343034303430343034300d0a

In compute node, from the blob, leaving the fixed length header part, I am able to convert bitmaps back to ascii-niebble-to Blob in the output message, in this way,ISO DFDL is correctly parsing the bitmaps..i.e RCD node is recieving the orignial data for bitmaps which is c2200000080000000080002008000002 in BLOB, then rest of the fields are also parsing succesfully except the ones which are AN like the field Reference_number_field37(12 AN EBCDIC; 12 bytes)

How to handle this efficiently? By usinge esql or is their anyway with in DFDL?


Last edited by mindspace on Wed May 02, 2018 10:12 am; edited 1 time in total
Back to top
View user's profile Send private message
timber
PostPosted: Tue May 01, 2018 2:22 pm Post subject: Reply with quote

Sentinel

Joined: 25 Aug 2015
Posts: 815

Quote:
How to handle this efficiently? By usinge esql or is their anyway with in DFDL?
You can do it either way.
a) Parse the bitmap as an array of hexBinary with representation='text' and write it out using a different DFDL model which has representation='binary'
b) extract the bytes using ESQL, do the conversion, write the BLOB to outputRoot.BLOB.BLOB and then parse using the standard DFDL model.

I've done it both ways - the choice is yours!
Back to top
View user's profile Send private message
mindspace
PostPosted: Tue May 01, 2018 4:03 pm Post subject: Reply with quote

Apprentice

Joined: 22 Jan 2013
Posts: 26

timber wrote:
Quote:
How to handle this efficiently? By usinge esql or is their anyway with in DFDL?
You can do it either way.
a) Parse the bitmap as an array of hexBinary with representation='text' and write it out using a different DFDL model which has representation='binary'
b) extract the bytes using ESQL, do the conversion, write the BLOB to outputRoot.BLOB.BLOB and then parse using the standard DFDL model.

I've done it both ways - the choice is yours!


Issue is not with bitmaps, i've followed the second(esql conversion) method as mentioned by you. But with the data fields, some data fields in the message are ebcdic blob and some or ascii blob.. please see the input message ....ive mentioned about field 37 which is ebcdic hex(F8F1F1F6F1F9F0F0F5F2F5F5-- F8 - 8, F1 - 1 etc) - ..how to handle such data using dfdl? I mean how to convert F8 to 8?
Back to top
View user's profile Send private message
timber
PostPosted: Wed May 02, 2018 1:24 am Post subject: Reply with quote

Sentinel

Joined: 25 Aug 2015
Posts: 815

Apologies - the long lines made the end of your last line invisible in my browser, so I missed the most important part of your question. Sounds as if you are simply asking how to convert a particular sequence of EBCDIC characters into a number.

Have you looked in detail at all of the number representations that are available in DFDL?
https://www.ibm.com/support/knowledgecenter/en/SSKM8N_8.0.0/com.ibm.dfdl.spec.doc/dfdl_13_6.htm

Looks like a textNumberRep="standard" with encoding set to the appropriate EBCDIC encoding name...but don't take my word for it.
Back to top
View user's profile Send private message
mindspace
PostPosted: Wed May 02, 2018 6:06 am Post subject: Reply with quote

Apprentice

Joined: 22 Jan 2013
Posts: 26

No luck, I tried different code pages but DFDL is not able to parse EBCDIC fields. I may be missing some thing here....

EBCDIC Hex values such as F8, F1 are being converted to BLOB in TCP input node as 4638, 4631, and same is being fed to RCD node DFDL, here I have set datatype to unsignedint, representation to standard, text and tried different encodings.
Back to top
View user's profile Send private message
timber
PostPosted: Wed May 02, 2018 6:21 am Post subject: Reply with quote

Sentinel

Joined: 25 Aug 2015
Posts: 815

Quote:
DFDL is not able to parse EBCDIC fields. I may be missing something here...
I can assure you that DFDL is perfectly capable of parsing and writing all kinds of EBCDIC fields. What you mean is that you have not been able to work out how to parse your EBCDIC numeric fields.
Quote:
EBCDIC Hex values such as F8, F1 are being converted to BLOB in TCP input node as 4638, 4631
I don't know what that means. Are you saying that a single byte with value 0xF8 (EBCDIC '8') is being passed to DFDL as two bytes with values 0x46, 0x38? Or are you just describing how the BLOB gets displayed in your trace file?
Either way, the TCPIPInput node is just emitting whatever arrives on the wire; it does not change the BLOB in any way.

Please answer that second question carefully - I cannot advise you unless I know exactly what byte values are being passed to DFDL.
Back to top
View user's profile Send private message
mindspace
PostPosted: Wed May 02, 2018 6:37 am Post subject: Reply with quote

Apprentice

Joined: 22 Jan 2013
Posts: 26

Thats right, from the debug mode, I can see that input F8 is represented as X'4638 in BLOB and same is being sent to DFDL.
Back to top
View user's profile Send private message
mindspace
PostPosted: Wed May 02, 2018 6:43 am Post subject: Reply with quote

Apprentice

Joined: 22 Jan 2013
Posts: 26

other thing I noticed is that if I pass 'F8' (instead of X'4638) in blob message(alter the blob in compute node) to DFDL and with proper IBM500 ebcdic encoding..DFDL is successfully parsing it to '8'
Back to top
View user's profile Send private message
mindspace
PostPosted: Wed May 02, 2018 7:49 am Post subject: Reply with quote

Apprentice

Joined: 22 Jan 2013
Posts: 26

I need to do some processing in esql same as bitmaps, so that actual data received (F goes to DFDL not X'4648, but for data fields in the message, which vary their positions and again not all fields are in EBCDIC..how to do this? so only specific fields if present must be converted like field_37
Back to top
View user's profile Send private message
timber
PostPosted: Thu May 03, 2018 2:59 am Post subject: Reply with quote

Sentinel

Joined: 25 Aug 2015
Posts: 815

It sounds as if some previous application has (possibly accidentally) expanded the EBCDIC digits into their hex-[b]ASCII[/i] representation and written the resulting ASCII string into the ISO8583 field. Nobody else represents text numbers like that, so your only option is to
- parse it as hexBinary using DFDL
- CAST the resulting BLOB to CHARACTER using ESQL
- CAST the resulting string to INTEGER using ESQL.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportMixed data inpu, ASCII and EBCDIC hex data parsing with DFDL
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.