| Author |
Message
|
| mandy13 |
 Posted: Mon Mar 10, 2014 7:36 am Post subject: Store msgId in database |
 |
|
Apprentice
Joined: 17 Apr 2013
Posts: 28
|
Hi,
I am trying to store msgId (message id) returned by MQGET in database using a c++ program. Can you tell me how can i effectively store this message id (MQBYTE24) using unixodbc.
I am doing a browse operation and storing the msgId's in database and selectively removing the messages using the msgId stored in database.
Thanks in advance |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Mon Mar 10, 2014 7:47 am Post subject: Re: Store msgId in database |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mandy13 wrote: |
Hi,
I am trying to store msgId (message id) returned by MQGET in database using a c++ program. Can you tell me how can i effectively store this message id (MQBYTE24) using unixodbc.
I am doing a browse operation and storing the msgId's in database and selectively removing the messages using the msgId stored in database.
Thanks in advance |
What have you tried? What were the results?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 10, 2014 8:43 am Post subject: Re: Store msgId in database |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mandy13 wrote: |
| I am doing a browse operation and storing the msgId's in database and selectively removing the messages using the msgId stored in database. |
Aside from the very valid questions of my associate, why are you doing this? It doesn't seem like a very efficient solution and will run like a dog with 3 legs on a queue with any depth (unless you're on z/OS, when it will run like a 4 legged dog with breathing problems).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mandy13 |
| Posted: Mon Mar 10, 2014 8:50 am Post subject: |
|
|
Apprentice
Joined: 17 Apr 2013
Posts: 28
|
| I stored the msgId in an unsigned char [24] and inserted in database using the format specifier %s but odbc gave an error saying end of string "'" could not be found in which msgId was enclosed. |
|
| Back to top |
|
|
| mandy13 |
| Posted: Mon Mar 10, 2014 9:01 am Post subject: Re: Store msgId in database |
|
|
Apprentice
Joined: 17 Apr 2013
Posts: 28
|
| Vitor wrote: |
| mandy13 wrote: |
| I am doing a browse operation and storing the msgId's in database and selectively removing the messages using the msgId stored in database. |
Aside from the very valid questions of my associate, why are you doing this? It doesn't seem like a very efficient solution and will run like a dog with 3 legs on a queue with any depth (unless you're on z/OS, when it will run like a 4 legged dog with breathing problems). |
I need to remove the message from MQ only if the processing is succesfull. In case of a crash we need to process the message again hence the browse operation and storing the message id in database to remove it from MQ after succesfull processing. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Mar 10, 2014 9:25 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mandy13 wrote: |
| I stored the msgId in an unsigned char [24] and inserted in database using the format specifier %s but odbc gave an error saying end of string "'" could not be found in which msgId was enclosed. |
MsgId and CorrelId are not character fields.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mandy13 |
| Posted: Mon Mar 10, 2014 9:42 am Post subject: |
|
|
Apprentice
Joined: 17 Apr 2013
Posts: 28
|
| bruce2359 wrote: |
| mandy13 wrote: |
| I stored the msgId in an unsigned char [24] and inserted in database using the format specifier %s but odbc gave an error saying end of string "'" could not be found in which msgId was enclosed. |
MsgId and CorrelId are not character fields. |
How do i store the MsgId? Do i need to convert it? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Mar 10, 2014 9:44 am Post subject: Re: Store msgId in database |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mandy13 wrote: |
I need to remove the message from MQ only if the processing is succesfull. In case of a crash we need to process the message again hence the browse operation and storing the message id in database to remove it from MQ after succesfull processing. |
Isn't that what syncpointing is for? Create a unit of work, if unsuccessful roll it back? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Mar 10, 2014 10:24 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mandy13 wrote: |
| bruce2359 wrote: |
| mandy13 wrote: |
| I stored the msgId in an unsigned char [24] and inserted in database using the format specifier %s but odbc gave an error saying end of string "'" could not be found in which msgId was enclosed. |
MsgId and CorrelId are not character fields. |
How do i store the MsgId? Do i need to convert it? |
In MQ-speak, MsgId and CorrelId are MQBYTE24 - 24 bytes of bits, not characters, not converted in any way. Pick another specification.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 10, 2014 10:50 am Post subject: Re: Store msgId in database |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mandy13 wrote: |
| I need to remove the message from MQ only if the processing is succesfull. In case of a crash we need to process the message again hence the browse operation and storing the message id in database to remove it from MQ after succesfull processing. |
And you decided not to use the built-in WMQ transaction control (which will automatically roll the message back on the queue to be reprocessed) because.....?
To return to your point:
- you can't hold the ids in any kind of character column. Locate a sutible binary column and use that
- this design will cost a fortune in resources as you're not only going to get poor response from the queue manager if the queue depth increases, you're going to get poor response from the DB (even if the queue depth is low) if the transaction rate is high.
Short answer - unless you have a low volume transaction model, this design is not going to perform very well at all. It also adds a dependency on a database table which is unnecessary and adds a point of failure.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Mon Mar 10, 2014 10:52 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Mandy, at first glance this does seem a very strange thing to do. As was mentioned earlier this is exactly what transactions are for.
Have your application use MQGMO_SYNCPOINT and only when all processing is completed do you issue MQCMIT(). Of course if you have database operations then you may wish to use a global transaction so that both the MQ system and your database are committed together.
Doing what you propose is much more complicated and very much less efficient. After all you must be writing the MsgId to the database outside of syncpoint (if you are concerned about a crash) which means a log force. If there isn't a crash then you need (I assume) to go and clear up these old database records which presumably is another log force.
Of course there could be perfectly valid reasons why you are doing what you propose that you just haven't told us about, it would be interesting to know a little bit more about your application and what lead you down this route before we suggest it is not the way to go,
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Mon Mar 10, 2014 7:08 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
Side stepping the discussion so far, MsgId can be stored in a 24 byte binary data type field in a database, or covert it to its hexadecimal print representation (48 characters 0-9 A-F) and store that.
_________________
Glenn |
|
| Back to top |
|
|
| mandy13 |
| Posted: Tue Mar 11, 2014 6:58 am Post subject: |
|
|
Apprentice
Joined: 17 Apr 2013
Posts: 28
|
Guys i totally agree to your point on syncpointing. But my application demands the following design.
Continuously browse the MQ and store the messages in an internal priority queue. The message that is picked for processing its MsgId is stored in database and deleted once the processing completes. If in case during processing the application crashes then i need to start processing from the same message as a part of recovery. So the database stores only 1 message id at a time.
Hex conversion did the trick. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Mar 11, 2014 8:00 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mandy13 wrote: |
| Guys i totally agree to your point on syncpointing. But my application demands the following design. |
Why? "Because that's the design I was told to use" is not a good answer.
| mandy13 wrote: |
| Continuously browse the MQ and store the messages in an internal priority queue. The message that is picked for processing its MsgId is stored in database and deleted once the processing completes. If in case during processing the application crashes then i need to start processing from the same message as a part of recovery. |
So if you're only browsing 1 message (and then storing it in a database), under what circumstances would you not browse the same message off post-crash? How could you get a different message and why would this matter to you if you did, given that the messages have no affinity? What advantage does the database give you and what will you do if the application is fine but the database is down / running slow?
| mandy13 wrote: |
| So the database stores only 1 message id at a time. |
Boy, that's going to thrash the db.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Mar 11, 2014 8:29 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
I suspect that the browsed message and MsgId stored in a DB is a part of a larger business process; and answers the question 'where was I?' if the flow is interrupted.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
