| Author |
Message
|
| DeadLetter |
 Posted: Tue Nov 29, 2016 3:54 am Post subject: Channel sequence number suddenly back to 1 |
 |
|
Novice
Joined: 15 Nov 2016
Posts: 11
|
Hi all,
Name: WebSphere MQ
Version: 7.1.0.6
Symptom:
Transmission Queue status: "retrying"
Cause:
Sending channel sequence number 176.359.742
Receiving channel sequence number 1
Workaround:
stop queue and reset sequence number on both to 1
Further analyses done:
Check log files on sending and receiving Q-Manager for first time of occurrence and check for root cause
Result:
none.
Question:
Is there any known issue with the mentioned version, that all of a sudden the sequence number gets reset to 1 w/o any further log record or side cause?
Thanks for any hint.
Carsten |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Tue Nov 29, 2016 4:54 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Ask the Admins at the receiving end what happened?
How do you know that it suddenly was reset?
Perhaps someone did a reset on the Receiver channel?
Perhaps the other end has failed over to a standby node? One that had not been used before?
If the other end can do that then you need to put in place procedures where the other Admins tell you that something like this has happened so that you can reset your sequence number.
Perhaps you are being subjected to a 'Man-in-the-middle' attack?
There is a lot of simple yet basic investigation to be done before asking if there is a product defect. From what information you have supplied us, it does not seem as if you have done that?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Nov 29, 2016 5:51 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Use runmqsc or MQExplorer to examine last chance date/time of receiving and sending channels. It is likely that the receiving end was deleted and redefined OR the RESET command was issued.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| smeunier |
| Posted: Wed Nov 30, 2016 12:01 pm Post subject: |
|
|
Partisan
Joined: 19 Aug 2002
Posts: 305
Location: Green Mountains of Vermont
|
This is a very timely question, as i was about to ask the same thing. We see this happen periodically one one of our channels, with no known reason that can be identified after checking for the obvious, resets, failovers, network outage, power failures, etc.
I have seen the receiver seq. # in the 800K range and then BOOM, the sender all of sudden is at 1. For me the issue seems to happen on RQSTR/SVR channels more than SDR/RCVR, but I suspect that is just coincidence. Is there a tried and true method to get to the root of why it happened. Every time we go through the same recourse with no explanation. Seems to be more internal, then external.
I'm about to change to a SDR/RCVR model, but I think that is a cosmetic change and eventually will see the same scenario. Getting to the root cause would be best, but not able to thus far.
for me, it occurs most on AIX MQ V7.5.0.6 high volume channel, but i just saw one from AIX -> z/os low volume. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Nov 30, 2016 1:21 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I would expect it to happen every time you change the ip of the server... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Nov 30, 2016 1:48 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| fjb_saper wrote: |
| I would expect it to happen every time you change the ip of the server... |
As I alluded to in my original post.
On one site I worked on we had to implement scripts to reset the channel sequence numbers when the main servers were moved to the DR site and again when it was moved back again.
IMHO, if this is the cause of the problem then it is NOT an MQ issue but a local admin issue.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| DeadLetter |
| Posted: Thu Dec 01, 2016 12:44 am Post subject: |
|
|
Novice
Joined: 15 Nov 2016
Posts: 11
|
Hi all,
thanks for the replies and "luckily" it seems, we are not the only ones experiencing this issue.
| Quote: |
Further analyses done:
Check log files on sending and receiving Q-Manager for first time of occurrence and check for root cause
Result:
none.
|
It was the first thought, that a failover had happened or the customer started a backup node o.s.
We found the first occurrence and checked the log files for corresponding events.
We also checked the statistics of the related items (channel, queue, manager) if there was any update done.
We also did check the syslog/message files of the server.
It really seems, that all of a sudden the receiver channel replaces the ticket roll and starts all over with 1.
It did record the event of RESET, when we initiated it to get back in sync on the connection. So it was not in a undocumented condition not able to log events. If something or somebody had issued a reset on the channel, it would have logged it from that perspective.
From the current point of view it does not seem to happen very often, so there is no "high impact" or and RTop ongoing for that, but it is indeed a strange unexplained event.
So I just was hoping anybody out there has had this too and might have a solution for it.
br
Carsten[/quote] |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Dec 01, 2016 3:58 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| DeadLetter wrote: |
It was the first thought, that a failover had happened or the customer started a backup node o.s.
We found the first occurrence and checked the log files for corresponding events.
We also checked the statistics of the related items (channel, queue, manager) if there was any update done.
We also did check the syslog/message files of the server.
It really seems, that all of a sudden the receiver channel replaces the ticket roll and starts all over with 1.
|
You should raise A PMR and get IBM's help to find out why this is happening.
If no human OR script (cron job) initiated the channel reset then there does seem to be a defect somewhere.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| jcv |
| Posted: Tue Dec 13, 2016 2:29 am Post subject: |
|
|
Chevalier
Joined: 07 May 2007
Posts: 411
Location: Zagreb
|
| smeunier wrote: |
We see this happen periodically one one of our channels, with no known reason that can be identified after checking for the obvious, resets, failovers, network outage, power failures, etc.
|
So, do you consider very temporary network outage to be a legitimate reason, or just a known reason that can cause channel sequence reset, or none of it? |
|
| Back to top |
|
|
| jcv |
| Posted: Tue Dec 13, 2016 8:25 am Post subject: |
|
|
Chevalier
Joined: 07 May 2007
Posts: 411
Location: Zagreb
|
| Could you please explain what exactly made you checking for the network outage as a root cause of your problem? Is it documented somewhere in KC? |
|
| Back to top |
|
|
|
|
