| Author |
Message
|
| sharadagarwal16 |
 Posted: Wed Mar 10, 2010 10:45 pm Post subject: Problem with message termination in TCPIPClientReceive Node |
 |
|
Novice
Joined: 04 Jun 2009
Posts: 13
|
Hi
I am in process to development of a flow which works as a tcpip client and it sends requests to a server socket and receives responses from that. The format of message is plain bit stream. Every part of it describes something. In first 8 bytes i got the message length which i need to read from the socket. My problem is that the responses neither have any termination character at the end nor these have any fixed length. Also the connection is not closed among more than one request-responses.
How can i decide that where i need to terminate my response message. As mentioned above, i can get the length of the response message (no of bytes) to read from the socket. But there is nothing in TCPIP nodes where i can dynamically set the bytes to be read from the socket.
Any suggestion in this is appreciated.
Thanks. |
|
| Back to top |
|
 |
| harish_td |
| Posted: Wed Mar 10, 2010 11:40 pm Post subject: |
|
|
Master
Joined: 13 Feb 2006
Posts: 236
|
|
| Back to top |
|
|
| sharadagarwal16 |
| Posted: Tue Mar 16, 2010 10:40 pm Post subject: |
|
|
Novice
Joined: 04 Jun 2009
Posts: 13
|
Hi
I have gone through the link and tried the same. But my problem is like the length of the rest of the message is coming in hexadecimal and it cannot be converted on the fly. So i am unable to give this field as length reference in the second field.
Any other way to achieve this..???
Or can we read from socket whatever is there on the socket irrespective of length or terminator or anything...using TCPIPClientReceive node? |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Mar 17, 2010 1:45 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| Sounds like ISO-8583. |
I agree, but sharadagarwal16 has not confirmed that yet.
| Quote: |
| In first 8 bytes i got the message length |
Please describe this 'length' in detail. Are you saying that the first 8 bytes are an integer value which gives the number of bytes in the remainder of the message? Or is the length just one of a number of fields in an 8-byte header?
| Quote: |
| there is nothing in TCPIP nodes where i can dynamically set the bytes to be read from the socket. |
You need to extract some data from the first 8 bytes, convert it to an integer value, and use that integer as the length of the message. In other words, the length is calculated by *parsing* the 8-byte header. Last time I looked there were some options related to parsing on the TCPIPInput node.
I suspect that you need to describe your data format using a TDS message set. But you have not provided enough information about the data format yet, so I can't say for sure. |
|
| Back to top |
|
|
| sharadagarwal16 |
| Posted: Wed Mar 17, 2010 2:20 am Post subject: |
|
|
Novice
Joined: 04 Jun 2009
Posts: 13
|
Yeah its similar to ISO-8583 but not exactly same. Its a propritory format in which first four bytes signifies start flag and next four gives length in hexadecimal. After these bytes there is a checksum of 8 bytes.
The whole message is a bit stream created by a legacy application. So i need to extract length from first 4 bytes add 8 to it and then need to receive those many bytes from the socket. e.g. first 8 bytes are 'SC'005C in which 'SC' is start flag and 005C = 92 is the length of the message. So i need to read 92+8=100 bytes from the socket. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Mar 17, 2010 3:35 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
You need to create a message set with a TDS physical format to describe your data format. The message definition does not need to be detailed - you only need to model the header and then model the remainder of the message as a field of type xs:binary whose length depends on the length field in the header.
Then you can use the 'Parsed Record Sequence' option to make the TDS parser to split up the stream for you.
If the message flow needs to process fields in the body of the message then obviously you may want to make the model more detailed. But it sounds as if you're just forwarding the message without changing it. |
|
| Back to top |
|
|
| sharadagarwal16 |
| Posted: Wed Mar 17, 2010 8:48 pm Post subject: |
|
|
Novice
Joined: 04 Jun 2009
Posts: 13
|
Thanks Kimbert...for your replies...
Yeah its fine that i need to create a message set with TDS physical format, but to specify the header as the length of second part of the message, the header must be of type:int, then only i can give it as a length reference, which is not true in my case. I did the same as you described, but failed because length is not coming in decimal(int) but hexadecimal, which need to be converted first in decimal.
Also I need to parse this message completely as i need to transform it to xml and then pass to the source system, which i can manage. At this moment, i am stuck with the length part.
Thanks... |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Mar 18, 2010 2:51 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Hold on...engage brain before proceeding. This gets a little 'hairy'.
The standard solution would be:
a) create an element 'Header' with a complex type
b) add elements to the complex type to describe its internal structure
c) set the TDS physical format properties on the child elements to make them parse correctly. This step would include setting the TDS Physical Type property of the length field to 'Integer' ( that answers one of your questions ).
d) add a simple element of type 'xs:hexBinary' and set its "Length Reference" property to point to Header.MessageLength.
Later on, you can change the xs:hexBinary to a complex type which describes the internal structure. This is just to get the basic scenario working.
However...you have a small problem:
| Quote: |
| i need to extract length from first 4 bytes add 8 to it and then need to receive those many bytes from the socket |
TDS can handle scenarios where the length of the referenced field is included in the length. That would cope with this:
| Code: |
length(Header.MessageLength) + Header.MessageLength |
but you need this:
| Code: |
| length(Header) + Header.MessageLength |
or this:
| Code: |
| length(Header.MessageLength) + Header.MessageLength + 8 |
There is a solution, though. I think you need to do it this way:
| Code: |
Element name="message"
complexType DataElementSeparation="FixedLength"
element name="Header"
complexType DataElementSeparation="FixedLength"
element name="startFlags" type="xs:int" length="4"
element name="MessageLength" type="xs:int" length="4" PhysicalType="Integer"
element name="checkSum" type="xs:hexBinary" length="8"
element name="AlmostAllOfTheMessageBody" type="xs:hexBinary" lengthReference="../Header/MessageLength"
element name="TheRemainderOfTheMessageBody" type="xs:hexBinary" length="8" |
That will allow you to use Parsed Record Sequence in the TCPIPInput node to split up the input stream.
In your message flow, you can concatenate the two BLOB fields "AlmostAllOfTheMessageBody" and "TheRemainderOfTheMessageBody" and then use CREATE...PARSE to parse the resulting BLOB using a more detailed message definition. |
|
| Back to top |
|
|
| sharadagarwal16 |
| Posted: Thu Mar 18, 2010 11:46 pm Post subject: |
|
|
Novice
Joined: 04 Jun 2009
Posts: 13
|
a) create an element 'Header' with a complex type
b) add elements to the complex type to describe its internal structure
c) set the TDS physical format properties on the child elements to make them parse correctly. This step would include setting the TDS Physical Type property of the length field to 'Integer' ( that answers one of your questions ).
| Quote: |
| Is it going to work if my value of of length is 005C? |
d) add a simple element of type 'xs:hexBinary' and set its "Length Reference" property to point to Header.MessageLength.
| Quote: |
| I am not able to give reference of length field here. As it only takes reference from siblings which are defined as int. |
Later on, you can change the xs:hexBinary to a complex type which describes the internal structure. This is just to get the basic scenario working. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Mar 19, 2010 12:40 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Please use the Preview button before you hit Submit. Your last post has the quotes and text switched around.
| Quote: |
| Is it going to work if my value of of length is 005C? |
What's special about the value 005C? Are you asking about the value 0x005C, or are you asking whether TDS can parse a binary integer? ( it can, and I explained that in my reply ).
| Quote: |
| I am not able to give reference of length field here. As it only takes reference from siblings which are defined as int. |
Did you actually read my reply? The length field *is* an integer. |
|
| Back to top |
|
|
| sharadagarwal16 |
| Posted: Fri Mar 19, 2010 5:19 am Post subject: |
|
|
Novice
Joined: 04 Jun 2009
Posts: 13
|
Hi Kimbert
I created the message set exactly as you explained. Even i did the same before you explained. You are not getting my problem. I mean to say here is the length itself is coming as hexadecimal. How can i parse it by defining the field as int. It gives me parsing error. Following is the sample message which can give you a better view of the problem.
| Code: |
| `SC`005C1.00CCB00000PPS 00000000DLGCONFFFF00000005TXBEG FFFFLIST PPS ACNTINFO:MDN=919164886193 A5BCC0AB |
In this message first 4 bytes i.e. 'SC' is start flag, next 4 bytes i.e. 005C defines length of rest of the message and last 8 bytes i.e. A5BCC0AB is checksum. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Mar 19, 2010 6:01 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
|
| Back to top |
|
|
|
|
