ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ API Support » Time based queue in .NET

Post new topic  Reply to topic
 Time based queue in .NET « View previous topic :: View next topic » 
Author Message
JoeConnolly
PostPosted: Thu Oct 24, 2013 1:11 pm    Post subject: Time based queue in .NET Reply with quote

Newbie

Joined: 10 Jul 2012
Posts: 2

What is the best way to implement a time based queue in MQ on .NET?

Each message must not be processed until a specific time.
Each message has an execution time between now and 60 minutes from now.
Assume multiple writers and readers.
We tried this in an Oracle database and it could not handle the load which is over 400 reads and writes per second.

The exact time at which the message can be processed is stored as a long message property of the form yyyymmddhhmmss.

Options considered include: MQ .NET API Browse and XMS

MQ .NET API using Browse

Write code snippet
Code:
MQMessage message = new MQMessage();
   message.Persistence = MQC.MQPER_PERSISTENT;
   message.WriteObject(object); //around 50k     
   message.SetInt8Property("RetryDateTime", GetTimestampAsLong(RetryDateTime));

   MQPutMessageOptions putMessageOptions = new MQPutMessageOptions();
   putMessageOptions.Options = MQC.MQPMO_FAIL_IF_QUIESCING | MQC.MQPMO_SYNCPOINT;
               
   mqConnection.Queue.Put(message, putMessageOptions);


Read code snippet

Code:
MQMessage message = null;
         try
         {
                mqConnection = GetConnection();
                MQGetMessageOptions getMessageOptions = new MQGetMessageOptions();
                getMessageOptions.Options = MQC.MQGMO_FAIL_IF_QUIESCING | MQC.MQGMO_NO_WAIT | MQC.MQGMO_BROWSE_FIRST | MQC.MQGMO_LOCK;
                long utcNow = GetTimestampAsLong(DateTime.UtcNow);

                // this loop will either find a message and break or throw a no msg found exception.
                while (true)
                {
                    message = new MQMessage();
                    mqConnection.Queue.Get(message, getMessageOptions);
                    long nextRetryDateTime = message.GetInt8Property("RetryDateTime");
                    if (nextRetryDateTime < DateTime.UtcNow)
                    {
                        getMessageOptions.Options = MQC.MQGMO_FAIL_IF_QUIESCING | MQC.MQGMO_NO_WAIT | MQC.MQGMO_MSG_UNDER_CURSOR;
                        mqConnection.Queue.Get(message, getMessageOptions);
                        break;
                    }
                    getMessageOptions.Options = MQC.MQGMO_FAIL_IF_QUIESCING | MQC.MQGMO_NO_WAIT | MQC.MQGMO_BROWSE_NEXT | MQC.MQGMO_LOCK;
                }

                myObject = (MyObject) message.ReadObject();
            }
            catch (MQException mqException)
            {
                if (mqException.Reason == 2033)  myObject = null;//MQRC_NO_MSG_AVAILABLE
                else throw;
            }



XMS using Message Selector

Code:
public override void Send(MyObject myObject)
        {
            IObjectMessage objectMessage = session.CreateObjectMessage();
            objectMessage.Object = myObject;
            objectMessage.SetLongProperty("RetryDateTime", GetTimestampAsLong(myObject.NextRetryDateTime));
            producer.Send(objectMessage);
        }

        public override myObject Receive(int timeout)
        {
            MyObject myObject = null;
            string selector = "RetryDateTime < " + GetTimestamp(DateTime.UtcNow);
         using(IMessageConsumer consumer = session.CreateConsumer(queue, selector))
            {
                IMessage message = consumer.Receive(timeout);
                if (message != null)
                {
                    IObjectMessage objectMessage = (IObjectMessage) message;
                    myObject = (MyObject) objectMessage.Object;
                }
            }

            return myObject;
        }


The MQ .NET API Browse option suffers from to many reads of the message when
there are many messages that are not ready to process.
Is there a way to just read the properties without reading the whole message?

The XMS version seems to work better but the rest of our MQ code uses MQ .NET and we use a connection string
containing multiple queue managers to provide a limited form of high availability.
Does XMS support a multiple queueManager connection string or do we have to implement it ourselves?
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Oct 24, 2013 2:54 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

No, MQ is designed to process messages as quickly as possible. You can code a delay using browse and intermediate databases but it is not likely to be very efficient or have high throughput.

Can the message producing application only put messages after the specified time, and let the normal MQ speedy processing occur after that?
_________________
Glenn
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Oct 25, 2013 5:33 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

You could of course place the messages on a dummy queue that is being browsed with an expiry with full content that sends them to the correct queue for immediate processing.

This would ensure that they are not being processed before expiry. It would not ensure that they are processed right after expiry...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
gbaddeley
PostPosted: Sun Oct 27, 2013 2:29 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

fjb_saper wrote:
You could of course place the messages on a dummy queue that is being browsed with an expiry with full content that sends them to the correct queue for immediate processing.
This would ensure that they are not being processed before expiry. It would not ensure that they are processed right after expiry...

fjb_saper,
That's a very creative use of Expiry and Report features of MQ. However, I wouldn't trust Expiry on my mission critical messages and then have Report message processing fail for some reason. If it fails, the mesasge is gone!
_________________
Glenn
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sun Oct 27, 2013 7:22 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

It was never an ideal solution in my mind. I think that the requirements should be revisited and the design changed... MQ is not a timer that allows you to store and process at a specific time...., I'm thinking here about a DB being browsed and acted upon at the critical time... Again I don't see this design working correctly in any volume based scenario...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
gbaddeley
PostPosted: Mon Oct 28, 2013 2:19 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

Quote:
What is the best way to implement a time based queue in MQ on .NET?

Joe, I think the answer is that there is no best way to implement this using MQ. For high performance I would look at using structure arrays or linked lists in application memory that could be ordered by time. This is the classic "dispatcher" approach used by operating systems. For reliability and recoverability, this should be backed up by a database.
_________________
Glenn
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » IBM MQ API Support » Time based queue in .NET
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.