| Author |
Message
|
| cr |
 Posted: Thu Oct 05, 2006 11:27 pm Post subject: Implementing a write lock on a message |
 |
|
Novice
Joined: 04 Oct 2006
Posts: 12
|
Hi
I'm trying to implement what I would describe as a write lock on a message in a queue. Must add that I have only been on the project involving MQ a week but I have read up and experimented with message options.
I'm so far able to browse and lock the message so other clients can no longer see it (2033 - no message). However, what I'd like to do is for other clients to be able to read that particular message while it's locked.
I suspect this functionality might be slightly at odds with the way MQ is intended to be used but this is the current requirement!
Is there a way to lock a message but still have it readable by other queue readers?
One idea: add another record, the lock, which clients must be nice enough to check exists before attempting to write a new entry? Not very robust!
Better way? Make a copy of the message, add "_ReadOnly" to its ID, and if clients can't find the the 'original' (as it's locked), let them fall back to using the read only version?
But in the latter case, a client can't discover that a new version exists without a blocking call (or a periodic polling) for that particular message ID. Is that correct?
I can appreciate these are all workarounds and so the implication might be that this requirement is a mismatch compared with what MQ is designed to provide.
If it helps anyone to provide suggestions, this particular message is a 'configuration' file which sits on a queue only accessed by various instances of an administrative tool (which would like to view the current configuration if it is locked).
Thanks very much to anyone who is willing to help with my first post! Off to work now.
C R |
|
| Back to top |
|
 |
| elvis_gn |
| Posted: Thu Oct 05, 2006 11:58 pm Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi cr,
U want your clients to only read(browse) a message, right ?
Firstly, How are u planning to let them edit a message lying in the queue ?
Regards. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Oct 06, 2006 1:43 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Just make the applications that need to receive the configuration use BROWSE to read the message.
Or create a very simple request/reply process that will send a copy of the config information to anyone who asks for it.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| cr |
| Posted: Fri Oct 06, 2006 4:32 am Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 12
|
Hi Jeff
There's one application but could be many instances - only one should be able to get 'write access' to the configuration message. Hence my use of lock.
However, other applications can no longer read the configuration message as MQ's implementation hides a locked message. So I don't think they can browse for it (no message will be the error returned). So how would a copy be returned also?
I think they can block and wait for the message to reappear can they not? I don't know whether this is the best way one would discover an update to that message. That is something else I have to investigate.
This is all new to me so help appreciated.
cr |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Oct 06, 2006 4:45 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
My request/reply idea is this.
Two queues - one that holds the config, and one that is the input queue.
ONE program only reads the config queue. It can even open and "lock" the message.
The same program reads the input queue - when someone asks for a copy of the config, the config server writes a copy of the config to the replyToQueue specified in the request message.
If someone needs to update the config, they can send a different request to the same input queue, and the config server can update the config.
But at this point you're really starting to use a queue as a database, and that's an anti-pattern in messaging.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| cr |
| Posted: Fri Oct 06, 2006 6:43 am Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 12
|
Jeff
Thanks for the dialogue.
If I understand your suggestion ... clients try and read the config queue.
If the message is there then they have write access and lock it and subsequently service the input queue.
If the message is not there then they can only read and so GET from the input queue. The client which first grabbed the config queue message can then PUT on the reply-to queue specified in the GET.
In order to receive updates and to potentially gain a write lock then clients that are reading could make a blocking wait call on the configuration queue.
So we essentially use two static queues and a dynamic reply-to queue.
I'm only trying to reiterate what you're saying in order to understand MQ idioms - as I say, have only been exposed to this technology within the last week. Am I on the right track here (as I would like to propose this idea)?
Is the following equivalent to this (albeit an even less messaging-like solution)?
Have a single queue containing the config message, and have a lock message. If the lock message is there, lock it. When you want to write the new config, GET the original, then PUT the new one back on.
I know what you're thinking, it seems against the nature of messaging ... browsing and locking and to update the configuration, getting it and putting a new one back on the same queue, with the same name!
Did I also mention that this queue will store old edits to the configuration message too :-)
Anyway, comments appreciated.
cr |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Oct 06, 2006 7:45 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
No.
Clients never try and read the config queue.
Clients send a message to a program. The program reads the config queue. The program sends a message back to the client, with the config. Clients read this message.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| cr |
| Posted: Fri Oct 06, 2006 10:51 am Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 12
|
Hi Jeff
Thanks for the message.
All the clients to the config queue are potential editors of the most recent config message. They're really all instances of the same admin tool.
I've inherited the requirement to make sure, out of all the different instances, only one of them has write access to the config.
The others must be able to read it until such time that the config is updated, which they need to know about and so reload or, if they can get write access in the first place, to be able to edit the config message and stop other instances being able to write to it.
This is my problem i.e. I cannot treat these clients differently. Do you think there is a nice solution?
Thanks,
cr |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Oct 06, 2006 3:11 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Have you thought about have a "lock" queue.
Does not matter what the "lock" message content is... Users can browse the config queue and view the message as much as they want.
If a user wants to update the config message he must first grab the message from the lock queue in a destructive mode. If he cannot somebody else is editing the message.
When done editing the config the user would:
in the same unit of work, write the message to the config queue and consume the old config message, and write a message back to the lock queue (thus releasing the lock on the config message). For notification purposes an additional publish message could be written to a specified topic. At that point in time a commit is called for.
To get the notification the interested parties would subscribe to the relevant topic....
Enjoy and have fun implementing this...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| cr |
| Posted: Wed Oct 11, 2006 2:15 pm Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 12
|
Hi
More requirements ...
Many instances of the same application need to read/write the Config (which is currently stored as a single message on a particular queue).
I've been told that, if an application holds on to (in some way i.e. 'locks') the Config for a certain amount of time, it should be automatically relinquished (as the user has gone away but the application is still around).
I can't use the MQ locking functionality on the Config message itself because it hides the message.
This is what I am thinking ...
The applications try and get a "Lock" message. If it is not there, they put one on the queue with an expiry. If it is there, they know they can just read the "Config" message. If the application holds onto the "Lock" for too long, it just disappears. Applications can then write a "Lock" if they want to.
All clients of this queue are well-behaved and know the protocol!
I know it's a bit back-to-front etc but this is what I have to go on:
There is one Config message.
It has to be locked if possible.
It can be read whilst locked.
Not allowed to incorporate any other queues, any publish-subscribe model, anything else apart from either locking a particular message (for a certain amount of time, which I don't think is possible from the API docs - please confirm that this is true) or implementing something equivalent to locking a particular message and every application (which we have control of) following the rules.
Help!
c r |
|
| Back to top |
|
|
| kevinf2349 |
| Posted: Wed Oct 11, 2006 2:25 pm Post subject: |
|
|
Grand Master
Joined: 28 Feb 2003
Posts: 1311
Location: USA
|
| hmmmm isn't this getting dangerous close to wanting to use MQ as a database? |
|
| Back to top |
|
|
| cr |
| Posted: Thu Oct 12, 2006 12:25 am Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 12
|
Yes, it is, but this is my task, unfortunately.
Unless I can present an alternative, that doesn't require too much extra infrastructure, then this is the current idea.
If I could find a way to have a lock expire, or similar, then that would be perfect!
cr |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Oct 12, 2006 2:18 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| cr wrote: |
If I could find a way to have a lock expire, or similar, then that would be perfect! |
If IBM had intended MQ to be a combination database / messaging system then perhaps they would have included the feature. Alas.... 
I've been faced with "ideas" like that in the past. They always seem to involve bending products in unintended directions don't they?
We march forwards, heads held high and hands tied behind our back. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 12, 2006 3:17 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
O.K. so let's look at your idea again:
- If the lock queue is empty put a msg on it with expiry (sufficient for task).
- Get the config message with browse option
- Modify the config message.
- start UOW
- check that lock is still there
- consume old config msg
- write new config msg
- consume lock msg
- commit and done
- If lock queue not empty (browse)
- check expiry left on lock message (browse should give that)
- poll lock queue untill empty
- start previous list.
Have fun implementing this
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Oct 12, 2006 3:27 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I continue to believe that the best way to handle this is by isolating the actual programs from the actual queue, using a request/reply paradigm.
If all of the clients send two kinds of messages - one to request the current config and one to update the current config - then the request queue itself will serve as your "synchronization" mechanism.
Your config server application reads one message at a time. It either replies with the current config, or updates the current config. It only does one operation at a time, so two update messages can't conflict.
Whether or not the current config is stored in a queue is then irrelevant to your admin applications, as they do not directly access the "storage" of the current config.
And you are still left with the same problem you had before, in that there does not exist any explicit mechanism for deciding which two "simulatneous" lock requests will actually acquire the lock - you can't necessarily control the order of the updates.
But you do have a lot more flexibility with this, though. You could for example have the admin applications send update messages with a higher (or lower!) priority than read messages. Then the config server would update the config before (or after!) pending reads .
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
