ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ API Support » Store msgId in database

Post new topic  Reply to topic Goto page 1, 2  Next
 Store msgId in database « View previous topic :: View next topic » 
Author Message
mandy13
PostPosted: Mon Mar 10, 2014 7:36 am    Post subject: Store msgId in database Reply with quote

Apprentice

Joined: 17 Apr 2013
Posts: 28

Hi,

I am trying to store msgId (message id) returned by MQGET in database using a c++ program. Can you tell me how can i effectively store this message id (MQBYTE24) using unixodbc.

I am doing a browse operation and storing the msgId's in database and selectively removing the messages using the msgId stored in database.

Thanks in advance
Back to top
View user's profile Send private message
bruce2359
PostPosted: Mon Mar 10, 2014 7:47 am    Post subject: Re: Store msgId in database Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

mandy13 wrote:
Hi,

I am trying to store msgId (message id) returned by MQGET in database using a c++ program. Can you tell me how can i effectively store this message id (MQBYTE24) using unixodbc.

I am doing a browse operation and storing the msgId's in database and selectively removing the messages using the msgId stored in database.

Thanks in advance

What have you tried? What were the results?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Mar 10, 2014 8:43 am    Post subject: Re: Store msgId in database Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

mandy13 wrote:
I am doing a browse operation and storing the msgId's in database and selectively removing the messages using the msgId stored in database.


Aside from the very valid questions of my associate, why are you doing this? It doesn't seem like a very efficient solution and will run like a dog with 3 legs on a queue with any depth (unless you're on z/OS, when it will run like a 4 legged dog with breathing problems).
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
mandy13
PostPosted: Mon Mar 10, 2014 8:50 am    Post subject: Reply with quote

Apprentice

Joined: 17 Apr 2013
Posts: 28

I stored the msgId in an unsigned char [24] and inserted in database using the format specifier %s but odbc gave an error saying end of string "'" could not be found in which msgId was enclosed.
Back to top
View user's profile Send private message
mandy13
PostPosted: Mon Mar 10, 2014 9:01 am    Post subject: Re: Store msgId in database Reply with quote

Apprentice

Joined: 17 Apr 2013
Posts: 28

Vitor wrote:
mandy13 wrote:
I am doing a browse operation and storing the msgId's in database and selectively removing the messages using the msgId stored in database.


Aside from the very valid questions of my associate, why are you doing this? It doesn't seem like a very efficient solution and will run like a dog with 3 legs on a queue with any depth (unless you're on z/OS, when it will run like a 4 legged dog with breathing problems).


I need to remove the message from MQ only if the processing is succesfull. In case of a crash we need to process the message again hence the browse operation and storing the message id in database to remove it from MQ after succesfull processing.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Mon Mar 10, 2014 9:25 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

mandy13 wrote:
I stored the msgId in an unsigned char [24] and inserted in database using the format specifier %s but odbc gave an error saying end of string "'" could not be found in which msgId was enclosed.

MsgId and CorrelId are not character fields.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
mandy13
PostPosted: Mon Mar 10, 2014 9:42 am    Post subject: Reply with quote

Apprentice

Joined: 17 Apr 2013
Posts: 28

bruce2359 wrote:
mandy13 wrote:
I stored the msgId in an unsigned char [24] and inserted in database using the format specifier %s but odbc gave an error saying end of string "'" could not be found in which msgId was enclosed.

MsgId and CorrelId are not character fields.


How do i store the MsgId? Do i need to convert it?
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Mar 10, 2014 9:44 am    Post subject: Re: Store msgId in database Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

mandy13 wrote:

I need to remove the message from MQ only if the processing is succesfull. In case of a crash we need to process the message again hence the browse operation and storing the message id in database to remove it from MQ after succesfull processing.

Isn't that what syncpointing is for? Create a unit of work, if unsuccessful roll it back?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
bruce2359
PostPosted: Mon Mar 10, 2014 10:24 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

mandy13 wrote:
bruce2359 wrote:
mandy13 wrote:
I stored the msgId in an unsigned char [24] and inserted in database using the format specifier %s but odbc gave an error saying end of string "'" could not be found in which msgId was enclosed.

MsgId and CorrelId are not character fields.


How do i store the MsgId? Do i need to convert it?

In MQ-speak, MsgId and CorrelId are MQBYTE24 - 24 bytes of bits, not characters, not converted in any way. Pick another specification.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Mar 10, 2014 10:50 am    Post subject: Re: Store msgId in database Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

mandy13 wrote:
I need to remove the message from MQ only if the processing is succesfull. In case of a crash we need to process the message again hence the browse operation and storing the message id in database to remove it from MQ after succesfull processing.


And you decided not to use the built-in WMQ transaction control (which will automatically roll the message back on the queue to be reprocessed) because.....?

To return to your point:

- you can't hold the ids in any kind of character column. Locate a sutible binary column and use that
- this design will cost a fortune in resources as you're not only going to get poor response from the queue manager if the queue depth increases, you're going to get poor response from the DB (even if the queue depth is low) if the transaction rate is high.

Short answer - unless you have a low volume transaction model, this design is not going to perform very well at all. It also adds a dependency on a database table which is unnecessary and adds a point of failure.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
PaulClarke
PostPosted: Mon Mar 10, 2014 10:52 am    Post subject: Reply with quote

Grand Master

Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand

Mandy, at first glance this does seem a very strange thing to do. As was mentioned earlier this is exactly what transactions are for.

Have your application use MQGMO_SYNCPOINT and only when all processing is completed do you issue MQCMIT(). Of course if you have database operations then you may wish to use a global transaction so that both the MQ system and your database are committed together.

Doing what you propose is much more complicated and very much less efficient. After all you must be writing the MsgId to the database outside of syncpoint (if you are concerned about a crash) which means a log force. If there isn't a crash then you need (I assume) to go and clear up these old database records which presumably is another log force.

Of course there could be perfectly valid reasons why you are doing what you propose that you just haven't told us about, it would be interesting to know a little bit more about your application and what lead you down this route before we suggest it is not the way to go,

Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
gbaddeley
PostPosted: Mon Mar 10, 2014 7:08 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

Side stepping the discussion so far, MsgId can be stored in a 24 byte binary data type field in a database, or covert it to its hexadecimal print representation (48 characters 0-9 A-F) and store that.
_________________
Glenn
Back to top
View user's profile Send private message
mandy13
PostPosted: Tue Mar 11, 2014 6:58 am    Post subject: Reply with quote

Apprentice

Joined: 17 Apr 2013
Posts: 28

Guys i totally agree to your point on syncpointing. But my application demands the following design.
Continuously browse the MQ and store the messages in an internal priority queue. The message that is picked for processing its MsgId is stored in database and deleted once the processing completes. If in case during processing the application crashes then i need to start processing from the same message as a part of recovery. So the database stores only 1 message id at a time.

Hex conversion did the trick.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Mar 11, 2014 8:00 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

mandy13 wrote:
Guys i totally agree to your point on syncpointing. But my application demands the following design.


Why? "Because that's the design I was told to use" is not a good answer.


mandy13 wrote:
Continuously browse the MQ and store the messages in an internal priority queue. The message that is picked for processing its MsgId is stored in database and deleted once the processing completes. If in case during processing the application crashes then i need to start processing from the same message as a part of recovery.


So if you're only browsing 1 message (and then storing it in a database), under what circumstances would you not browse the same message off post-crash? How could you get a different message and why would this matter to you if you did, given that the messages have no affinity? What advantage does the database give you and what will you do if the application is fine but the database is down / running slow?

mandy13 wrote:
So the database stores only 1 message id at a time.


Boy, that's going to thrash the db.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Mar 11, 2014 8:29 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

I suspect that the browsed message and MsgId stored in a DB is a part of a larger business process; and answers the question 'where was I?' if the flow is interrupted.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » IBM MQ API Support » Store msgId in database
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.