| Author |
Message
|
| mpong |
 Posted: Tue Nov 12, 2013 4:43 am Post subject: Additional Instances |
 |
|
Disciple
Joined: 22 Jan 2010
Posts: 164
|
Hi,
One of the flow which receives 30K to 50 k messages every hour has an 10 additional instance on MQInput Node, the flow does only INSERT and UPDATE of all fields, we noticed that some of the messages are neither inserted nor updated in DB, randomly messages are skipped out without any exception, Please let me know your comments on db operation when there is multi instance.
Sample Msg: AAAAAAA$BBBBBB$CCCCCC$DDDD$EEEEE |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Nov 12, 2013 4:51 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
What data have you assembled that shows any connection between the number of additional instances and the fact that sometimes messages are not processed?
What data have you assembled that shows any description of what happens when messages are not processed? |
|
| Back to top |
|
|
| mpong |
| Posted: Tue Nov 12, 2013 5:19 am Post subject: |
|
|
Disciple
Joined: 22 Jan 2010
Posts: 164
|
Hello Jeff,
It is already running on AIX in PD, source has increased more no of message hence we suspect this, there is a java program which reads each line from the file and put into queue. I wont be able to see connection between the number of additional instances. I am trying to run the flow on my local system. Please suggest. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Nov 12, 2013 5:57 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
How comprehensive is your error handling?
could you be experiencing DB deadlocks?
Are there any errors in the log files?
Is there any path through your flow that a message might take where it does nothing and basically 'drops off the end of the world'?
Have you captured a set of messages and run them through your test/dev system? If so are they showing the same problem?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mpong |
| Posted: Tue Nov 12, 2013 6:58 am Post subject: |
|
|
Disciple
Joined: 22 Jan 2010
Posts: 164
|
Hi, there is no DB exception or DB deadlocks, everything looks fine, it is only single select, update and insert query, since there are multiple threading(additional instances) on database select,update and insert is causing the issue  ?, test/dev system it worked fine too. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 12, 2013 7:22 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mpong wrote: |
| Hi, there is no DB exception or DB deadlocks |
Is this because there are no errors, or your error handling is insufficient to trap them?
| mpong wrote: |
| it is only single select, update and insert query, since there are multiple threading(additional instances) on database select,update and insert is causing the issue |
So your DB can't handle multiple threads and is deadlocking. Given that all WMB is doing is running the same single threaded processes that works fine as many times as there are instances.
How do you know this "java program" reliably reads each line from the file & puts it to the queue? Why isn't the flow reading the file directly?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bielesibub |
| Posted: Tue Nov 12, 2013 9:33 am Post subject: |
|
|
Apprentice
Joined: 02 Jul 2008
Posts: 40
Location: Hampshire, UK
|
| Quote: |
| test/dev system it worked fine too |
For how many requests?
Have you checked the DLQ? - assuming the queue manage has one configured. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Nov 12, 2013 10:32 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
One last point that can get overlooked
Have you tested the error handling in the flow and especially for errors on every one of the DB calls.
Why not add something to the code that logs 'I'm done ok and here is a bit of the message so that it can be identified'
ALL and I repeat ALL critical flows should log the input and output messages. If there is no output then logging things like the primary key (or something else significant) should be logged on the way out.
Just enforcing this simple rule makes bug finding a whole lot easier. It also allow you to point the finger at the systems that send data to Broker.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
|
|
