| Author |
Message
|
| sarasu |
 Posted: Tue Oct 08, 2013 8:07 am Post subject: Repeating MsgID during MQPut |
 |
|
Master
Joined: 02 Feb 2006
Posts: 229
|
All,
We are getting the MsgID repeating during the MQput for DataBase update.
When we send few messages this is not happening. but when we perform load test, hitting with 10K messages, we are seeing this.
is there a fix for it ?
We are using MQ on z/linux.
Name: WebSphere MQ
Version: 7.0.1.9
And Broker version:
MQSI 8.0.0.2
Thanks |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Tue Oct 08, 2013 8:11 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Would you care to show us all the error messages?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Oct 08, 2013 8:16 am Post subject: Re: Repeating MsgID during MQPut |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sarasu wrote: |
| is there a fix for it ? |
Probably. How does your code link an MQPut with a database update & what part does the message id play?
How does your code create the message id? Specifically are you sure that each message from each source uses the one generated by WMQ and doesn't set it itself?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sarasu |
| Posted: Tue Oct 08, 2013 9:40 am Post subject: |
|
|
Master
Joined: 02 Feb 2006
Posts: 229
|
Here we have an auto generated LogID for each transaction.
And using that LogID, we are updating the database in all 4 loggings we have.
we have total 6 calls to DB from the message flow.
4 inserts and 2 updates based on the auto generated LogID.
++++++++++
The errors we are getting are, when we tried to update he LogID for the last table out of four loggings we have, we are getting the same LogID getting updated. Which is because fo the same MsgID, we are getting from the MQ Message.
Let me know if it clarifies your questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Oct 08, 2013 10:09 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sarasu wrote: |
| Here we have an auto generated LogID for each transaction. |
Auto generated from what? The message id? Be specific.
| sarasu wrote: |
| Which is because fo the same MsgID, we are getting from the MQ Message. |
I repeat, what generates the message id? WMQ? The application?
| sarasu wrote: |
| Let me know if it clarifies your questions. |
No, it doesn't.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sarasu |
| Posted: Tue Oct 08, 2013 10:42 am Post subject: |
|
|
Master
Joined: 02 Feb 2006
Posts: 229
|
I don't want to confuse you. Just forget everything mentioned above.
To be precise,
Message flow placing a message in to the queue. For each message there is a MsgID associated with it.
When we hit 10K messages, is there a possibility of repeating this MsgID ? if yes, how to avoid this.
Thanks |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Oct 08, 2013 10:45 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| sarasu wrote: |
I don't want to confuse you. Just forget everything mentioned above.
To be precise,
Message flow placing a message in to the queue. For each message there is a MsgID associated with it.
When we hit 10K messages, is there a possibility of repeating this MsgID ? if yes, how to avoid this.
Thanks |
If QMGR generates the MsgId, it is guaranateed to be unique as part of the value is the system clock.
| Quote: |
| A MsgId generated by the queue manager consists of a 4-byte product identifier (AMQ␢ or CSQ␢ in either ASCII or EBCDIC, where ␢ represents a blank), followed by a product-specific implementation of a unique string. In WebSphere® MQ this contains the first 12 characters of the queue-manager name, and a value derived from the system clock. All queue managers that can intercommunicate must therefore have names that differ in the first 12 characters, in order to ensure that message identifiers are unique. The ability to generate a unique string also depends on the system clock not being changed backward. To eliminate the possibility of a message identifier generated by the queue manager duplicating one generated by the application, the application must avoid generating identifiers with initial characters in the range A through I in ASCII or EBCDIC (X'41' through X'49' and X'C1' through X'C9'). However, the application is not prevented from generating identifiers with initial characters in these ranges. |
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Oct 08, 2013 10:56 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sarasu wrote: |
Message flow placing a message in to the queue. For each message there is a MsgID associated with it.
When we hit 10K messages, is there a possibility of repeating this MsgID ? |
If the id is generated by the queue manager, no.
If the id is generated by the code you've placed in the flow, yes.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Oct 08, 2013 11:17 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
It appears that one possible solution is to make sure that the MsgId is unique even if the Application Generates it. It is fairly easy to do.
One Flow I have worked on used Flow generated MsgID's because we wanted to do an MQget maybe a day later to remove that message and only that message from the queue.
The used a combination of the either the whole of parts of the following
Dealer ID
Make ID
Model ID
VIN Number
Date Produced
Over more than 10Million messages passed through the system during test we never had a duplicate MsgId.
By using this construct another flow could generate the Id in order to get the message via ID using the MsgId. An Auto generated Id would not work in this situation but you have to take a lot of care in designing the way the ID is constructed.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Oct 08, 2013 11:22 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| smdavies99 wrote: |
It appears that one possible solution is to make sure that the MsgId is unique even if the Application Generates it. It is fairly easy to do.
One Flow I have worked on used Flow generated MsgID's because we wanted to do an MQget maybe a day later to remove that message and only that message from the queue.
The used a combination of the either the whole of parts of the following
Dealer ID
Make ID
Model ID
VIN Number
Date Produced
Over more than 10Million messages passed through the system during test we never had a duplicate MsgId.
By using this construct another flow could generate the Id in order to get the message via ID using the MsgId. An Auto generated Id would not work in this situation but you have to take a lot of care in designing the way the ID is constructed. |
While there may be some value in having user data embedded into the MsgId, if your only concern is uniqueness, one can use UUID as an application-generated MsgId taking care to pair the 32 bytes down to 24 bytes by omitting the MAC address portion.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| sarasu |
| Posted: Tue Oct 08, 2013 11:43 am Post subject: |
|
|
Master
Joined: 02 Feb 2006
Posts: 229
|
Thanks for the inputs.
Once MsgID got generated, we are making it as below in two different compute nodes.
| Code: |
SET OutputLocalEnvironment.Variables.BLOB = InputLocalEnvironment.WrittenDestination.MQ.DestinationData.msgId;
|
and
| Code: |
DECLARE MSGIDBLOB BLOB InputLocalEnvironment.Variables.BLOB;
DECLARE MSGIDCHAR CHAR CAST (MSGIDBLOB AS CHAR CCSID 437); |
Is there a possibility of having duplicates now ? or I am still good not having duplicates of MsgId.
Thanks |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Oct 08, 2013 11:53 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
You really have to go back to the originator of the MsgId and see how they generate it.
If they let MQ generate it then there is very little chance but as has already been said if it is being generated by the application then there might well be chance of duplicates unless, and as has once again been pointed out, you are very careful in how the MsgId is being created.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Oct 08, 2013 12:00 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sarasu wrote: |
| Is there a possibility of having duplicates now ? |
You're casting a byte array into a character. There's a serious posibility that 1 or more of the bytes will be unprintable in the 819 code page and map to a space or a period. Hence while the WMQ generated byte level msg id is unique, the character representation of it may not be.
And if the original id is not WMQ generated, all bets are off.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sarasu |
| Posted: Tue Oct 08, 2013 12:10 pm Post subject: |
|
|
Master
Joined: 02 Feb 2006
Posts: 229
|
Correct. In our case, WMQ is only generating the MsgID for us.
But the CHAR representation of the MsgID is getting regenerated.
| Code: |
Examples of our MsgID in BLOB and CAHR respectively.
414d512050444c4e5831314d303030325251318f2004830d
AMQ PDLNX11M0002RQ1 ƒ
|
As mentioned above 819 code page is not printable. Do you suggest any other code pages to use here in order to print all characters ? |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Oct 08, 2013 12:56 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sarasu wrote: |
| Do you suggest any other code pages to use here in order to print all characters ? |
The only thing which can "print" every possible hex representation is 16 bit Unicode & I'd be impressed if the database table you're storing this in supports that as defined. You may be able to have that switched. This may cause more problems than it fixes, if possible.
You should take a step back & ask what value (no pun intended) this id offers you. Why are you storing messages in a database based on the character representation of the message id? What use will this be when you try to read the message back from the database for whatever purpose you store this message for? Typically the message id only has value inside WMQ; how does this suit your requirement better than (for example) a unique id generated by the database software?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
