| Author |
Message
|
| manojtare |
 Posted: Tue Nov 24, 2020 1:29 am Post subject: MQ OPEN 2045 error |
 |
|
Novice
Joined: 24 Nov 2020
Posts: 11
|
Hi,
My application in Cobol on ZOS tries to open an MQ queue with INPUT-SHARED and MQOO-FAIL-IF-QUIESCING options. The next command would have been MQGET. However, the MQOPEN fails intermittently about 10% of times i.e. if the application tries to open the queue, 90 out of 100 times it works but fails with 2045 error for the rest 10 times.
The message is put by a Java application accessing the queue directly using IP/port.
The queue is local and MQ is installed on mainframe. It was recently upgraded to v9.1.
Does anybody have a clue here? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Nov 24, 2020 5:37 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Is the "local" queue in a queue sharing group?
Does any other application get from that queue?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Tue Nov 24, 2020 9:46 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Is it possible that you have a bug in your code. Are you really presenting the same options each time ? Perhaps you would like to post the code fragment of the open call just to get a second pair of eyes on it.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| manojtare |
| Posted: Tue Nov 24, 2020 7:41 pm Post subject: |
|
|
Novice
Joined: 24 Nov 2020
Posts: 11
|
Hi,
I dont think there is any bug here but here is the code anyway
| Code: |
compute OPTIONS = MQOO-INPUT-SHARED +
MQOO-FAIL-IF-QUIESCING
call 'MQOPEN' using HCONN,
OBJECT-DESCRIPTOR,
OPTIONS,
Q-HANDLE,
COMPLETION-CODE,
REASON
|
And it is possible that the queue is in a queue sharing group. I have already asked the team handling it to check. And not sure if any other application also uses this queue. That is also being checked. Sometimes, people just try to reuse! But will that cause this problem?
Thanks for the responses! |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Tue Nov 24, 2020 7:47 pm Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
No I don't see any bug there. One of the issues with COBOL is that it is not very good at bitwise OR so people write X + Y rather than X | Y. This is fine but can cause problems if in a loop and you end up adding more than once. However, you don't seem to do that. So, the next question I suppose is whether the queue opened is always the same ? Is it always a local, sharable queue ?
Regards,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| manojtare |
| Posted: Tue Nov 24, 2020 7:55 pm Post subject: |
|
|
Novice
Joined: 24 Nov 2020
Posts: 11
|
Hi Paul,
Thanks for a quick response!
The queue is always the same, it is a local queue and defined with SHARE option. Even the default input open option is share and not exclusive.
We are suspecting that the upgrade to MQ v9.1 caused a problem. The queue is erroneously defined with trigger options even if no triggering is used. So perhaps MQ is taking extra time to do some processing and at the same time the queue is being accessed by the code. However, this has been working for years so we need to check if that indeed solves the problem. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Tue Nov 24, 2020 8:11 pm Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Well sorry but I have run out of ideas. The next things I would be doing, if it were me, is trying to reproduce the problem with an MQ trace on hope that the trace gave more information about why the generic reason code 2045 is returned.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Nov 24, 2020 10:11 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
You are showing "compute" options. You are not showing how the options are being initialized.
As Paul said, depending on the initialization of the options and the number of times the assignment is being made, it may be possible that some memory value from a previous assignment lingers.
Please make sure you correctly initialize the options EVERY time. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| manojtare |
| Posted: Tue Nov 24, 2020 11:24 pm Post subject: |
|
|
Novice
Joined: 24 Nov 2020
Posts: 11
|
I will try to get the trace enabled perhaps.
About the initialization, I doubt that will happen because it works perfectly in other environments. But no harm to check it anyway.
Thanks for the ideas guys. |
|
| Back to top |
|
|
| manojtare |
| Posted: Tue Nov 24, 2020 11:26 pm Post subject: |
|
|
Novice
Joined: 24 Nov 2020
Posts: 11
|
Sorry one more related question. I am no expert on MQ so that's why... Which log on mainframe can tell me which options were used to do the MQOpen. Will I not get that in case of an error, unless there is a trace?
And if I have to ask for a log, which log will it be? I have received some MSTR and CHIN log but that didnt help. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Nov 25, 2020 3:38 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Remember COBOL overwrites a lot of stuff and does not initialize a field (options) unless you do so explicitly. So you may have bits that are there just from a previous call and not necessarily the same call.
As for the trace the best would be for your program to output the options value right before the call. This way you can check.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| tczielke |
| Posted: Wed Nov 25, 2020 6:20 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
What picture clause did you use to define your OPTIONS working storage variable? Is it an 01 level?
What environment is this happening in? CICS? Batch? Also, is your COBOL application multi-threaded?
There is a z/OS MQ API trace that can capture the control block of the MQOD that is passed from your COBOL program to the MQ address space. That would be a good way to track down the issue.
The other thing that would be helpful is to capture the contents of the OPTIONS variable before it is passed into the MQOPEN and its contents after the MQOPEN returns a 2045. If the OPTIONS value has changed in that before and after comparison, that would clearly point to an IBM MQ issue. I am pretty sure the OPTIONS variable is being passed by reference into the MQOPEN call, so the IBM MQ code has access to your OPTIONS working storage field although they should not be changing it. If the OPTIONS variable still has the good value after the 2045 is returned, that would also be evidence that the issue is on the IBM MQ side.
Do you always set the OPTIONS field right before the MQOPEN? I have seen COBOL application issues in the past where application array overlays stepped on DB2 parameter control blocks in the working storage section and caused sporadic DB2 errors that were unexplainable to the DB2 and application team. If you set the OPTIONS field and then later do another MQOPEN assuming the OPTIONS field was unchanged, you may want to add some type of debug statement to see if the field was overlaid. That is what we had to do to get to the bottom of the issue.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
| manojtare |
| Posted: Thu Nov 26, 2020 9:37 pm Post subject: |
|
|
Novice
Joined: 24 Nov 2020
Posts: 11
|
Additional information. Before this MQOpen call, the program opens another queue with MQOO-OUTPUT option
| Code: |
compute OPTIONS = MQOO-OUTPUT +
MQOO-FAIL-IF-QUIESCING
call 'MQOPEN' using HCONN,
OBJECT-DESCRIPTOR,
OPTIONS,
Q-HANDLE,
COMPLETION-CODE,
REASON |
and puts a message on that. And now with this MQOpen call for input, it is trying to get the reply. All this happens in the same program. Note that the first MQPut call is with MQPMO-NO-SYNCPOINT option so that the message is sent straightaway. Also note that the same OPTIONS variable is used for both MQOPEN calls (and this has been working since ages).
So, if something should have been overwritten, the previous call should also have failed. But that doesnt happen.
The options input-shared/output etc are defined at 10 level, part of a group variable for all such MQ constants. The storage prior to the MQOO-INPUT-SHARED is all such constants. So if the storage was overwritten, the previous call should also have failed.
Also, OPTIONS variable is defined as
| Code: |
01 OPTIONS pic 9(9) binary value 0.
|
This is a CICS environment.
Abut MQ API trace, do you mean the one described here https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.tro.doc/q039670_.htm
If not, can you please guide me?
The OPTIONS are set just right before the MQOPEN as I have shared before. And as far as I have seen across the system, they are always set right before.
About changing the code to add some additional trace, it's a bit difficult because it's a production code and they dont want to make additional changes just before Christmas freeze period. So if we can find out another solution, it would be great. The MQ trace should come in quite handy.
|
|
| Back to top |
|
|
| tczielke |
| Posted: Fri Nov 27, 2020 7:05 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
Yes, that is the z/OS MQ API trace. It should trace the MQ open options value that is being passed into the MQOPEN call. It might be better here to have a PMR opened with IBM and see what other trace/dumps they might want captured here to debug this sporadic 2045 error. Even in a Prod environment, the mainframe has a lot of sophisticated problem determination tools (e.g. SLIP TRAP) that can be used to help get to the bottom of the issue.
For the CICS region that this application runs in, is there transaction isolation tuned on? If yes, that would protect and rule out one transaction from stepping on another transaction's user storage (i.e. COBOL working storage). However, the issue reported here does not sound like random corruption from another CICS transaction.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Nov 27, 2020 2:46 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
The options field is not initialized before the Compute Options command.
This means that any value could be part of the field. There is no telling what the resulting value is going to be as there is no telling what the initial value of the field is before the COMPUTE command.
There is also no telling whether the Storage the OPTIONS field is declared in is thread safe.
The best bet would be to output the Options field after the Compute Command and before the CALL 'MQOPEN' command.
This would definitely show us what the options are.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
