| Author |
Message
|
| Vitor |
 Posted: Thu Jul 05, 2012 10:15 am Post subject: |
 |
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
| If you're here for alternate solutions, they must have pictures of you in compromising positions or something. |
Me, compromising positions? It's never happened. In front of a camera. That survived the accident. I think they have pictures of my manager's manager in a compromising position. 
More seriously, they don't report to my manager's manager's manager; the chain of command converges at a point where you sometimes see the faces on the annual report but will never meet them. Since both my manager's manager's and his manager are unprepared to go head to head with the fiefdom that contains the WMQ team for political reasons I'm stuck here.
(The 3rd stage manager is question was described to me as "lazy", "cheap" & "paranoid" so you're pretty close).
I envy the ability of another regular poster to influence very senior management. Rest assured, posting here for alternate solutions was a last resort after all attempts to change direction politically failed.
I draw comfort from this morning's status conference call, where my manager's manger opened the meeting with:
| Quote: |
| Throttleing WMB. I'm quite sure <Vitor's real name> would prefer to throttle me at this point and I can understand why, but it is what it is. |
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
 |
| Nikswe |
| Posted: Thu Jul 05, 2012 1:09 pm Post subject: Re: WMB Speed Control |
|
|
Newbie
Joined: 28 May 2012
Posts: 9
Location: Stockholm
|
I have been enjoying the thread. Thanks!
| Vitor wrote: |
| About twice a day, a file of around 90k records is placed onto the broker's server & processed via a FileInput node. |
Isn't this the main problem? Is it not possible for the sending system to create the files with records more often to get a more spread out payload?
But I assume not, since you came here for advice on slowing down the broker.
/Nikswe |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jul 05, 2012 6:46 pm Post subject: Re: WMB Speed Control |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Nikswe wrote: |
I have been enjoying the thread. Thanks!
| Vitor wrote: |
| About twice a day, a file of around 90k records is placed onto the broker's server & processed via a FileInput node. |
Isn't this the main problem? Is it not possible for the sending system to create the files with records more often to get a more spread out payload?
But I assume not, since you came here for advice on slowing down the broker.
/Nikswe |
And it is not possible to set a different priority for batch and online? That too would solve quite a number of problems / questions... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 06, 2012 5:32 am Post subject: Re: WMB Speed Control |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fjb_saper wrote: |
| And it is not possible to set a different priority for batch and online? That too would solve quite a number of problems / questions... |
It's not a question of jamming up the system with batch records so the online doesn't process.
The root problem is that I recieve a 90k record file, which WMB will chew through in X seconds generating 90k WMQ messages. Given that it takes X*10 for the downstream process to chew through all of these records many of these records will sit on the queue for a period of time. Which politically they many not do.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 06, 2012 5:34 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| Vitor wrote: |
| I was told "we don't review designs, we're not the message police"....) |
Right. So they're not the message police, so they don't have any say in how many messages pile up in queues. |
But they are the queue police. So they have no control (becuase they've relinquished it) over the message behaviour or their content but retain control over the queues and their administration.
Yes, I know, I called it that as well and worse. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jul 06, 2012 5:59 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| You do remember that Broker flows can issue PCF commands, right? |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Jul 06, 2012 6:56 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| mqjeff wrote: |
| You do remember that Broker flows can issue PCF commands, right? |
That is really underhand..... 
The PHB who don't like queues would have a fit if they saw this happening but hey, we aren't going to tell now are we?
I can't help wonder in these 'numpties' haven't reduced the max Q depth everywhere to the absolue minimum just to save a small bit of disc space.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Jul 06, 2012 7:09 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| smdavies99 wrote: |
| just to save a small bit of disc space. |
Disc space is the least expensive resource a company will ever buy. Just think about the payroll cost to add more human resources to fix an application that would have not mis-behaved if it had proper disc space.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 06, 2012 7:09 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| You do remember that Broker flows can issue PCF commands, right? |
I did know that, but had forgotten.
It's ingenious, but has 2 issues in this scenario:
1) Broker here can't issue PCF commands - it's not a member of mqm & has no access to the command queue
2) I can affect max depth like this but can't affect the depth monitoring software which will grass me up.
But it's a valid point. I'm not dealing with putting "too many" messages on a queue; I'm dealing with them knowing I've done it....
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 06, 2012 7:11 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| smdavies99 wrote: |
| just to save a small bit of disc space. |
Disc space is the least expensive resource a company will ever buy. Just think about the payroll cost to add more human resources to fix an application that would have not mis-behaved if it had proper disc space. |
And it's not even about buying it; the stated objection is to the amount of administration required to get it mounted on the queue manager's server. I am 75% confident there's purchased but unallocated disc in the farm.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mgk |
| Posted: Fri Jul 06, 2012 8:19 am Post subject: |
|
|
Padawan
Joined: 31 Jul 2003
Posts: 1642
|
So, have you thought about writing each record to its own file with a fileoutput node, rather than to a queue. Then, in a second flow which starts with a Timer node read one file (in a FileRead node) and put that on the real queue. Rinse and repeat...
Kind regards,
_________________
MGK
The postings I make on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Jul 06, 2012 8:27 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| mgk wrote: |
| Rinse and repeat... |
Thats how the Cobol programmer died in the shower. He was following the directions on the shampoo bottle and got caught in an infinite loop.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 06, 2012 8:34 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mgk wrote: |
| So, have you thought about writing each record to its own file with a fileoutput node, rather than to a queue. Then, in a second flow which starts with a Timer node read one file (in a FileRead node) and put that on the real queue. Rinse and repeat... |
Not in terms of using a file; thank you.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Jul 06, 2012 9:10 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Assuming their q that only allows a few messages on it lives on QM2
And your message flow runs on a Broker connected to QM1
You could set up a dedicated SNDR / RCVR channel pair between the two QMs for this, and set the Message Retry Interval and Message Retry Count paramaters on the RCVR channel so that when the destination queue fills up (Max Depth is reached, no matter how low they make it), the SNDR/RCVR channel will keep working on slipping new messages into the their queue as space in the q depth allows. Meanwhile the queuing occurs in the XMITQ on QM1, where disk space is abundant and sanity rules.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 06, 2012 9:51 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
| Meanwhile the queuing occurs in the XMITQ on QM1, where disk space is abundant and sanity rules. |
Sanity does not rule; they own that queue manager as well. Key reason the broker's not in the mqm group for it's own queue manager.
Good idea though.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
