ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexChallenge ForumChallenge Question - 07 / 2008

This forum is locked: you cannot post, reply to, or edit topics.This topic is locked: you cannot edit posts or make replies. Goto page Previous  1, 2
Challenge Question - 07 / 2008 View previous topic :: View next topic
Author Message
Challenger
PostPosted: Tue Jul 15, 2008 8:41 am Post subject: disk mirroring Reply with quote

Centurion

Joined: 31 Mar 2008
Posts: 115

Yes, good insight for a solution ....

except the Challenge problem states that the two sites are outside a metro cluster distance apart which is the limit of normal disk mirroring solutions.
Back to top
View user's profile Send private message
mq_developer
PostPosted: Sun Jul 27, 2008 2:51 pm Post subject: Reply with quote

Voyager

Joined: 18 Feb 2002
Posts: 82

Why dont have another queue defined that will hold "COD" messages from the input queue.

And go about using COD message as the state of the last successfully processed transaction and continue to read from their on the DR Side.

Contents of COD queue (last successfully processed msg in this case 100 ) will be available to other site as part of replication or and as an failsafe over replication , you can also define a static route (remote queue to DR) from PROD to DR. (in this case program reading off Input queue has to send COD message to Distribution list - one to local queue , another to remote queue going to DR)

So you have two queues where state of the last succefully processed transaction is available, assuming message IDs are defaulted , least of the MsgId's between 2 queues should be a good failsafe starting point.

For housekeeping COD messages in the queues , they can be placed with expiry - so there will not be a continuous pile up of messages
Back to top
View user's profile Send private message
Challenger
PostPosted: Wed Jul 30, 2008 9:50 am Post subject: cod solution Reply with quote

Centurion

Joined: 31 Mar 2008
Posts: 115

Will need synchronization utilities at DR site that would not be too complex.

They would need to read/get down the replicated input Q to point in Q where timestamp => COD replicated message timestamp ... if I understand your example ... seems it would work. Actually sounds a lot like a proposed solution by a vendor !

Also because there is no guarantee two RMs (Q and DB) will have their data replicated in synchrony no quarantee Q replication will match DB replication data on DR site after disaster occurence. COD message may get sent but DB replication data block corresponding to it may not, for example.

Any possible solution that would not need such synchronization utilities between Qs RM and DB RM ?

Overall ,though, Good idea looks workable ...
Back to top
View user's profile Send private message
mq_developer
PostPosted: Mon Aug 04, 2008 3:53 pm Post subject: Reply with quote

Voyager

Joined: 18 Feb 2002
Posts: 82

There are 3 references (COD Queue , Data Queue , Dabatase) - depending on how duplicates will be handled , least of the above 3 references needs to be taken for processing. But yes this app will be complex , tools like ICS (now process server) , make data synchronization (though they are ideally used between two systems) so easy ..

PS : Dont have any vendor affiliation - lone entity !
Back to top
View user's profile Send private message
Challenger
PostPosted: Wed Aug 06, 2008 10:06 pm Post subject: Reply with quote

Centurion

Joined: 31 Mar 2008
Posts: 115

Closing Remarks, Conclusions and Winner Announcement:

The July challenge problem perhaps more simply stated would be as follows:

1.) At primary site maintain synchronization between messages on Q and messages updated into DB
2.) At DR site achieve as close as possible a mirror image of this(primary site) Q store and DB store environment.

The need for HA at the primary and DR sites while a real world requirement does not add or subtract any useful details in resolving the nexus of the actual problem challenge.

Putting ones finger on just what is, is not simple!

For example:
If one focuses only on replicating messages from the Primary to the DR site you have recovery ability of source data for replay, but what do you replay all some or none of the messages. Problem as stated was minimize duplicate updates, as well as lost source messages as in both cases either source messages may not in all cases be recoverable and in some cases duplicates(updates) can be problematic. So need to minimize both as close to zero as possible. And oh, by the way minimize stat up time as close to zero as possible. So just replaying all messages and not doing DB replications is out.

If one focuses only on replicating updated DB table data from the Primary site to DR site you have updates to the DB but do you have all, some or none from a given interval. And if some or none where/how to get those you havent updated into DB?

So of course one must replicate both.

But , and here is one crucial point to grasp the problem, they are completely independent replications!
But, and here is another crucial point to grasp of the problem, they must be replicated in synchrony! Or one must have a means to hopefully easily and speedily synchronize the Q store and DB store at the remote DR site.

One responder who clearly seemed to have her finger on it, suggested that one just uses disk volume mirroring and leave the details up to the sophisticated raid devices and algorithms available that do such things today.
Excellent idea and would likely work but for the challenge problem condition that the DR site is beyond the distance of what is todays distance limitation for disk mirroring using dark fiber which is ~80 Km or ~62 miles.

So where to go next?
Remember to synchronize the Primary site DB updates from the Q back at the DR site, the updated messages have to be in some way reflected at the DR site. In other words the gets need to be replicated some how. Not an easy thing to do asynchronously in WMQ!

One responder clearly also had his finger on it again, however, with his suggested use of COD messages. His idea was along the line that would result from the 2PC function of GETting a message off the source queue and updating the DB at Primary site and have the COD message from the GET reply back or be replicated back to the DR site for processing by a pre-startup utility that would drain a replicated Q to the point in time matching latest COD message!!! Clever I would say. And likely doable without too much coding pain as I said.

For grasping the problem in its fullest and giving a workable solution I would say mq_developer gets the mug.

The actual solution taken by the customer in question was to use a different and to their thinking simpler and all around better solution. A solution where the Q store and DB store had to be in synchrony from Primary to DR site because the Queuing system had its Q store in the Database. Since they were/are an Oracle shop they used Oracles AQ for messages and replicated the databases (both application and messages) with Oracles DATAGUARD product.

Though IBM on previous versions of WAS allowed a DB as store for messages current versions do not. Also though competitors like Tibcos EMS allow an option to persist messages in a DB store, WMQ at this time does not.
So though the solutions given may work they still appear not to be optimal to a user looking at the total solution space.

Regards,
July Challenger
Back to top
View user's profile Send private message
Display posts from previous:
This forum is locked: you cannot post, reply to, or edit topics.This topic is locked: you cannot edit posts or make replies. Goto page Previous  1, 2 Page 2 of 2

MQSeries.net Forum IndexChallenge ForumChallenge Question - 07 / 2008
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.