| Author |
Message
|
| Carla Viragh |
 Posted: Mon Jan 19, 2004 9:51 am Post subject: Messages (storage) |
 |
|
Voyager
Joined: 31 Oct 2003
Posts: 92
Location: São Paulo - Brasil
|
Hi all.
It´s interesting... I have read some months ago that messages are stored only in log files but... If I need to find it in log, I can´t because log is impossible to read, right?
Today I was looking for a message and I found it inside the file C:\Arquivos de programas\IBM\WebSphere MQ\Qmgrs\QM1\QUEUES\QLOCAL!QM1!TESTE\Q
Great... but when I put another message, it is inside my queue but it appears inside the file a lot of minutes late. Why this happens? Is it normal? There is another way, another log that I can identify if the message was delivered and received? (Not using COA and COD, only using MQ). For example, if someone says that message ID XXXYYY was sent to me, and I´d like to know if it is realy here in my QM.
I only know the message was delivered if programs ask COA, COD?
Windows 2000.
_________________
Carla Viragh |
|
| Back to top |
|
 |
| JasonE |
| Posted: Mon Jan 19, 2004 9:58 am Post subject: |
|
|
Grand Master
Joined: 03 Nov 2003
Posts: 1220
Location: Hursley
|
| Messages are not accessible outside the MQ api. Messages are split across the logs (unreadable format!) and the Q files. Even the format of the queue files are unusual, and it is not possible to recover messages from just a Q file or just the logs (or both!). |
|
| Back to top |
|
|
| Carla Viragh |
| Posted: Mon Jan 19, 2004 10:10 am Post subject: |
|
|
Voyager
Joined: 31 Oct 2003
Posts: 92
Location: São Paulo - Brasil
|
You mean the only way to know if a message was delivered to my QM is the program asking for a COA, COD? Just a message report can prove that a message has delivered?
And... why the message has a big delay to appear inside the Q file?
_________________
Carla Viragh |
|
| Back to top |
|
|
| mqonnet |
| Posted: Mon Jan 19, 2004 11:03 am Post subject: |
|
|
Grand Master
Joined: 18 Feb 2002
Posts: 1114
Location: Boston, Ma, Usa.
|
Well, since this topic could be huge, lets revisit basics.
When you issue MQPUT, your message would either be put or it would fail. In case MQ successfully PUTS the message onto the desired queue, it would return success, else it would return a reason code which one could use to figure out what went wrong. And i think this is the first, best and the easiest way to confirm that the message actually was put successfully. How it was stored and where it was stored depend on the message attributes on the put. If you asked for a persistent message, it would be stored on disk as opposed to Non-persistent which would be stored in memory.
From Designers and programmers perspective, the only reason one would usually want to even go that route is to design/allocate enough storage space(logs). Thats it. Once thats done, both designers and developers could just write apps to put/get messages without really having to worry about the underlying process that MQ adheres to.
Of course, using Transactions and persistent messages would guarantee you the delivery of the message. You could as well use reports to keep track of messages though.
Not sure if this helps in your investigation, but thats my 2 cents.
Cheers
Kumar |
|
| Back to top |
|
|
| EddieA |
| Posted: Mon Jan 19, 2004 11:26 am Post subject: |
|
|
Jedi
Joined: 28 Jun 2001
Posts: 2453
Location: Los Angeles
|
Carla,
For the first question, yes. But only for the eventual destination QM. For your local QM, a return/reason of 0/0 is enough to know that the message is safe. Unless you don't trust MQ itself,  a message report will let you know that the message either arrived or was consumed. Unless you have a business need for this confirmation, then just knowing it was given to MQ should be enough.
Next. Was the message persistent. If not, then it might never appear in the Q file, as there is no reason to 'harden' it. For persistent, maybe MQ just writes it to the log initially, to make it recoverable, and then waits to see if it is immediately read by an application or MCA before using the Q file. Only after a delay, is it then hardened to the Q. Just *my* 2 pence, as I've no knowledge of how the internals actually work. 
Cheers,
_________________
Eddie Atherton
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0 |
|
| Back to top |
|
|
| bower5932 |
| Posted: Mon Jan 19, 2004 1:22 pm Post subject: |
|
|
Jedi Knight
Joined: 27 Aug 2001
Posts: 3023
Location: Dallas, TX, USA
|
| I remember a performance discussion at one of the conferences that talked about how frequently messages that were just written were gotten. One of the ways that was used to make things faster was to keep these messages in memory in the hopes that somebody would get it right away and save the write to disk. |
|
| Back to top |
|
|
| Carla Viragh |
| Posted: Tue Jan 20, 2004 6:36 am Post subject: |
|
|
Voyager
Joined: 31 Oct 2003
Posts: 92
Location: São Paulo - Brasil
|
Thanks for all assistance! You are great!
Kumar,
My problem is far from MQ... I know how it works and I realy trust it 
What is happening here is: My "partner" says that I did not send a message to him and I know that my application returned SUCCESS and no message is on the DLQ, well... the message was gone... Fine, but my partner insists that he didn´t receive this! I junt want to prove him that I sent this message. So, I can´t see it in log and I junt can see the text (application data) inside the Q file, the message ID or destination QM, date, hour, nothing else I can see about the message to prove to my fool partner  We don´t use messages report because this is a test project.
I would like to know if this (COD COA) will be the only way to prove that I sent the message and he can not cry if he can not find it!
Well, Eddie said Yes to this question.
Second question:
Bower and Eddie said exactly what I thought ... I think Q file only receives data if no application takes the message in X minutes.
(I was testing only persistent messages).
Sorry for my bad english, I hoppe you can understand my real doubt 
_________________
Carla Viragh |
|
| Back to top |
|
|
| bower5932 |
| Posted: Tue Jan 20, 2004 6:51 am Post subject: |
|
|
Jedi Knight
Joined: 27 Aug 2001
Posts: 3023
Location: Dallas, TX, USA
|
| If you are just testing, you can always shut down the getting application. You can then use the amqsbcg program to show that your messages made it to the queue. However, once you get away from testing, this won't leave any proof of the fact that you sent the message. |
|
| Back to top |
|
|
| mqonnet |
| Posted: Tue Jan 20, 2004 7:15 am Post subject: |
|
|
Grand Master
Joined: 18 Feb 2002
Posts: 1114
Location: Boston, Ma, Usa.
|
Well, as has already been mentioned here, and as you also know, COA/COD is the best way to prove your point here.
But again, if your put app comes back with a success and your message is persistent, then you can convince your partner that the message was actually put. Since MQ guarantees delivery and this being a Persistent message there is now way this is getting lost or something.
Also as bower pointed out, a browse on the queue where your message landed up would also prove that your message did land up.
To further enhance this you could put the message with specific values of msgid, priority etc to prove even further that this is for sure your message only.
But as bower pointed out, all this would just be for discussion sake, in real world all of this is "assumed". Since MQ is relied for its guaranteed delivery.
Hope this helps.
Cheers
Kumar |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jan 20, 2004 8:26 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| mqonnet wrote: |
| But again, if your put app comes back with a success and your message is persistent, then you can convince your partner that the message was actually put. Since MQ guarantees delivery and this being a Persistent message there is now way this is getting lost or something. |
It's very easy for this message to get lost, even if it is persistant.
The other side merely has to throw it away, and not log or admit the fact that they did so. "Yes, I know you say that your side did not delete these messages, and that nothing on your side could delete them. But we never got it, so it's your problem until you can show otherwise."
COA/D is not the only way to prove that the message didn't get lost before it got to the other side, but it's the easiest. Another way would be a channel exit that logged msg ids that pass through the channel.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mqonnet |
| Posted: Tue Jan 20, 2004 8:55 am Post subject: |
|
|
Grand Master
Joined: 18 Feb 2002
Posts: 1114
Location: Boston, Ma, Usa.
|
Jeff, could you please give me some examples where MQ would "throw" away a Persistent Message "without logging" anywhere.
From what i understand this is just not possible. There must be some logging some place even if the message is deleted/lost/thrown away.
The only possibility would be "pull the plug" scenario where a possible loss of data could occur. But again, if your message is persistent, then MCA would put this in a UOW and which would again mean, that the message would not be committed and backed out, and message still is in the xmitq.
If it were a direct put to a local queue, the whole scenario gets even simpler.
Cheers
Kumar |
|
| Back to top |
|
|
| Carla Viragh |
| Posted: Tue Jan 20, 2004 9:14 am Post subject: |
|
|
Voyager
Joined: 31 Oct 2003
Posts: 92
Location: São Paulo - Brasil
|
Yes Jeff, in my production environment I have a exit that logs the messages. (And messages reports too)...
After all these posts I am convinced that there is no other way to prove that the message was gone.
Thanks!
(And I have the same doubt as Kumar, Jeff)
_________________
Carla Viragh |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jan 20, 2004 9:26 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| mqonnet wrote: |
Jeff, could you please give me some examples where MQ would "throw" away a Persistent Message "without logging" anywhere.
From what i understand this is just not possible. There must be some logging some place even if the message is deleted/lost/thrown away. |
I'm not saying MQ would throw the message away and not log the fact that it did so. I'm saying the partner would throw the message away, and not log or admit the fact.
And again, as JasonE said
| Quote: |
| Messages are split across the logs (unreadable format!) and the Q files. Even the format of the queue files are unusual, and it is not possible to recover messages from just a Q file or just the logs (or both!). |
So if MQ logs anything in the case of an API call that destroys a message, it's not useful.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mrlinux |
| Posted: Tue Jan 20, 2004 10:34 am Post subject: |
|
|
Grand Master
Joined: 14 Feb 2002
Posts: 1261
Location: Detroit,MI USA
|
As for your delays at seeing them in the queue file, if they are non-persistent messages they will be stored in ram until the default 64kb buffer is filled.
_________________
Jeff
IBM Certified Developer MQSeries
IBM Certified Specialist MQSeries
IBM Certified Solutions Expert MQSeries |
|
| Back to top |
|
|
| Carla Viragh |
| Posted: Tue Jan 20, 2004 12:02 pm Post subject: |
|
|
Voyager
Joined: 31 Oct 2003
Posts: 92
Location: São Paulo - Brasil
|
OK... but the messages are all persistents. Messages non persistent never appears on Q file.
I think this delay is part of MQ structure, it works like this, maybe to save time in case of applications getting messages faster. But it doesn´t make sense if we think a little bit... If MQ structure has this delay, why it needs to write to Q file, for security? Probably not! The message is safe in log files... so... strange!
But, let MQ in peace whith its Qs files
_________________
Carla Viragh |
|
| Back to top |
|
|
|
|
