| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| 'C' Client Receives Embedded NULLS In The Reply Message |
« View previous topic :: View next topic » |
| Author |
Message
|
| vlt2010 |
 Posted: Thu Jan 28, 2010 11:50 am Post subject: 'C' Client Receives Embedded NULLS In The Reply Message |
 |
|
Newbie
Joined: 28 Jan 2010
Posts: 2
|
The Environment:
MQ V6.0.2.5 Client on SunFire running Solaris 8/SunOS 5.8.
IBM Global Security Kit is used for SSL connectivity and there is a Capitalware security exit.
The MQ client application is 'C' source compiled with Sun Studio 11.
The application uses basic MQPUT and MQGET logic.
The message is put on the queue with format MQFMT_STRING and expects a reply MQMT_REQUEST
The get message option MQGMO_CONVERT is used with MQENC_NATIVE encoding and character set MQCCSI_Q_MGR.
The program works without issue 99% of the time.
On occasion the reply message contains embedded NULLS in the application data (This has been confirmed using the MQ tracing program 'strmqtrc').
My counterparts on the server side have verified that the NULLS/low-values are being placed in the message.
When this happens the MQGET fills the buffer with data only up to the first NULL.
I have tried numerous things within the application to get all of the data but without success.
An obvious solution would be to modify the server application so that NULLS are not embedded in the message but there are other non-technical factors that make this easier said than done.
The question is:
From the client's perspective, given the environment described, can anything be done in the client application or with the MQ Client software to get all the data returned in the reply?
I have searched the publib boulder ibm infocenter site, mqseries net forum archives and FAQ's, the WWW, the MQ manuals and looked at the sample applications that came with the client installation. I have found some information that pertains to Java applications but nothing that addresses this particular scenario.
My specialty is contact center software and not MQ but I volunteered to set up the MQ client environment and write the code so now I own it. It's a low budget operation!
We have a MQ Java client that handles embedded NULLS and returns all off the data but unfortunately because of some proprietary issues the UI application that receives the data cannot interface with Java without some expensive licensing. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Jan 28, 2010 11:59 am Post subject: Re: 'C' Client Receives Embedded NULLS In The Reply Message |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Nice problem description! 
| vlt2010 wrote: |
| From the client's perspective, given the environment described, can anything be done in the client application or with the MQ Client software to get all the data returned in the reply? |
I would suspect that, because the data is in a format of STRING and the client is written in C that WMQ is indeed returning all the data; it's the C application that's trying to read it as a string. Because a string in C is delimited by a null character, it's perfectly reasonable for it to assume the first null it comes across is the end of the string. This would also explain why the Java works as (so far as I know) Jave doesn't have null-terminated strings.
Someone will probably correct me on that. 
The best recourse is to use the message length returned by the get. This will give you the actual length of data in the message, from which you can retrieve the data using non-string techniques.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| vlt2010 |
| Posted: Thu Feb 11, 2010 10:32 am Post subject: Solution |
|
|
Newbie
Joined: 28 Jan 2010
Posts: 2
|
Thanks for the nudge Vitor!
I wanted to monitor this in production for a while before posting the fix.
The solution is pretty simple but it is only meant to deal with a specific null data issue (we had no others):
.
.
.
| Code: |
MQCHAR GetMsgBuffer[BUFFER_LENGTH]; /* get message buffer */
MQLONG GetMsgLen; /* length of message received */
/* additional code begins */
char *buffPtr; /* message buffer pointer used when null encountered */
char tmpBuf[BUFFER_LENGTH]; /* temporary message buffer used when null encountered */
int buffLen; /* holds length value to be used when null encountered */
/* additional code ends */ |
.
.
.
MQGET
.
.
.
| Code: |
GetMsgBuffer[GetMsgLen] = '\0'; /* null terminate the get message buffer */
/* additional code begins */
buffLen = strlen(GetMsgBuffer); /* determine the string length of the get message buffer */
if ((strcmp(MQObj->tranKey, "GAI")==0)
&& (buffLen < GetMsgLen)) /* for specific transactions when above statement results */
/* in a value less than the message length returned by MQGET */
{
buffPtr = GetMsgBuffer; /* set buffer pointer to the value of get message buffer */
strcpy (tmpBuf, buffPtr); /* copy buffer pointer contents to the temporary message buffer */
buffPtr +=buffLen+2; /* increment the buffer pointer by bufflen plus 2(nulls) */
strcat (tmpBuf, " "); /* concatenate two spaces to make up for the 2 nulls */
strcat (tmpBuf, buffPtr); /* concatenate incremented buffer pointer */
strcat (tmpBuf, "\0"); /* append a null terminator */
buffLen = strlen(tmpBuf); /* determine the string length of the temporary message buffer */
memcpy (&MQObj->returnText[0],
&tmpBuf, buffLen); /* copy the temporary buffer to calling application return buffer */
}
else /* else there was no length discrepancy */
{
/* additional code ends */
memcpy (&MQObj->returnText[0],
&GetMsgBuffer, GetMsgLen); /* copy the get message buffer to calling application return buffer */
} |
.
.
.
|
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Feb 11, 2010 2:22 pm Post subject: Re: Solution |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| vlt2010 wrote: |
| Code: |
if ((strcmp(MQObj->tranKey, "GAI")==0)
&& (buffLen < GetMsgLen)) /* for specific transactions when above statement results */
/* in a value less than the message length returned by MQGET */
{
buffPtr = GetMsgBuffer; /* set buffer pointer to the value of get message buffer */
strcpy (tmpBuf, buffPtr); /* copy buffer pointer contents to the temporary message buffer */
buffPtr +=buffLen+2; /* increment the buffer pointer by bufflen plus 2(nulls) */
strcat (tmpBuf, " "); /* concatenate two spaces to make up for the 2 nulls */
strcat (tmpBuf, buffPtr); /* concatenate incremented buffer pointer */
strcat (tmpBuf, "\0"); /* append a null terminator */
buffLen = strlen(tmpBuf); /* determine the string length of the temporary message buffer */
memcpy (&MQObj->returnText[0],
&tmpBuf, buffLen); /* copy the temporary buffer to calling application return buffer */
} |
|
Why don't you deal with all possible nulls in all possible msgs, rather than dealing with a specific case and playing around with strcat?
| Code: |
GetMsgBuffer[GetMsgLen] = '\0'; /* null terminate the get message buffer */
buffLen = strlen(GetMsgBuffer); /* determine the string length of the get message buffer */
if ( buffLen < GetMsgLen ) /* for string length less than the message length returned by MQGET */
{
for( i = GetMsgLen-1 ; i >=0 ; i-- )
if( GetMsgBuffer[i] == NULL )
GetMsgBuffer[i] == ' ';
}
memcpy (&MQObj->returnText[0],
&GetMsgBuffer, GetMsgLen); /* copy the get message buffer to calling application return buffer */ |
_________________
Glenn |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 11, 2010 3:59 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| Why don't you deal with all possible nulls in all possible msgs... |
Good catch! It is possible that nulls may be interspersed with other non-null application data - not just trailing. Nulls are not so much an issue for other programming languages.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Fri Feb 12, 2010 4:20 pm Post subject: Re: Solution |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| gbaddeley wrote: |
| Code: |
GetMsgBuffer[GetMsgLen] = '\0'; /* null terminate the get message buffer */
buffLen = strlen(GetMsgBuffer); /* determine the string length of the get message buffer */
if ( buffLen < GetMsgLen ) /* for string length less than the message length returned by MQGET */
{
for( i = GetMsgLen-1 ; i >=0 ; i-- )
if( GetMsgBuffer[i] == NULL )
GetMsgBuffer[i] == ' ';
}
memcpy (&MQObj->returnText[0],
&GetMsgBuffer, GetMsgLen); /* copy the get message buffer to calling application return buffer */ |
|
Of course that should be
| Code: |
| GetMsgBuffer[i] = ' '; |
_________________
Glenn |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sun Feb 21, 2010 5:58 pm Post subject: Re: 'C' Client Receives Embedded NULLS In The Reply Message |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| vlt2010 wrote: |
| On occasion the reply message contains embedded NULLS in the application data |
In C, NULL is typically a macro, defined in various headers as a null pointer constant. It should not be confused with the null character.
| vlt2010 wrote: |
| When this happens the MQGET fills the buffer with data only up to the first [null character] ... can anything be done in the client application ... ? |
MQGET returns MQBYTEs. As Vitor correctly pointed out, it's more likely that the client application stops reading the data after the first null character. Either replace the function calls and code that expect null-terminated strings, with function calls and code that don't, (as Vitor suggested), or alter the message data, so that it doesn't contain null characters.
| vlt2010 wrote: |
| Code: |
| /*(1)*/ MQCHAR GetMsgBuffer[BUFFER_LENGTH]; /* get message buffer */ |
... MQGET ...
| Code: |
/*(2)*/ GetMsgBuffer[GetMsgLen] = '\0'; /* null terminate the get message buffer */
/*...*/
/*(3)*/ strcat (tmpBuf, "\0"); /* append a null terminator */
/*...*/
/*(4)*/ memcpy (&MQObj->returnText[0],
&tmpBuf, buffLen); /* copy the temporary buffer ... */
/*...*/
/*(4)*/ memcpy (&MQObj->returnText[0],
&GetMsgBuffer, GetMsgLen); /* copy the get message buffer ... */ |
|
- Why not eliminate GetMsgBuffer (and tmpBuffer), and have MQGET write directly to MQObj->returnText? Copying large buffers can get expensive.
- If GetMsgLen == BUFFER_LENGTH, then appending a null character here will cause a buffer overflow.
To prevent this, the MQGET BufferLength parameter needs to be less than BUFFER_LENGTH.
- strcat(tmpBuffer, "\0") is a no-op; it doesn't append a null character.
- For &MQObj->returnText[0] the & and [0] cancel each other out; MQObj->returnText works just as well.
For &tmpBuffer and &GetMsgBuffer, the & is superfluous; these are already pointers.
memcpy() does not copy the terminating null character; was that intentional?
As gbaddeley pointed out, a simple for-loop can replace null characters with spaces:
| Code: |
for (int i = 0; i < GetMsgLength; i++)
if (MQObj->returnText[i] == '\0')
MQObj->returnText[i] = ' '; |
But a call to memchr() can probably do it more efficiently:
| Code: |
// MQObj->returnText points to MQBYTE array with at least BUFFER_LENGTH bytes
// MQLONG DataLength, remainingLength;
// MQBYTE * startPtr; /* starting point for null character scan */
// MQBYTE * foundPtr; /* location of found null character */
/* MQGET(..., BUFFER_LENGTH, MQObj->returnText, &DataLength, ...); */
remainingLength = DataLength; /* remaining bytes to scan */
startPtr = MQObj->returnText; /* starting point for scan */
while (remainingLength > 0
&& (foundPtr = memchr(startPtr, '\0', remainingLength)) != NULL)
{
*foundPtr = ' '; /* replace null with space */
foundPtr++; /* skip to next byte, for next scan */
remainingLength -= foundPtr - startPtr;
startPtr = foundPtr;
} |
|
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
