| Author |
Message
|
| mkitclr |
 Posted: Fri May 31, 2013 10:59 am Post subject: Volume testing |
 |
|
Novice
Joined: 19 May 2011
Posts: 16
|
hi,
we are trying to test the Pre-prod env with million records of prod.
to acheive this we tried qload(MO03) and q utility(we could able to successfully load the Prod data to a file and ftped the file to pre-prod and tried to load pre-prod Queue, which failed(no problem in FTPing))..they are failing on huge volumes. so we thought we could setup a communication b/w prod and pre-prod.
Prod and Pre-prod Queue manager names are same(different hosts  )
when trying to put a message on remote queue of prod we are getting Unknown Remote Queue Manager error(2087).
Question: can queue managers with the same name can communicate?
-Is there a utility to handle that huge volumes?
Appreciate ur help. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Fri May 31, 2013 11:06 am Post subject: Re: Volume testing |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mkitclr wrote: |
| Prod and Pre-prod Queue manager names are same |
This is a very, very, very bad idea. One trivial configuration error and you'll find WMQ's name resolution sending pre-production data to your live system.
| mkitclr wrote: |
| Question: can queue managers with the same name can communicate? |
No, because the name resolution will tie itself in knots as you've seen. You seriously need to fix this, it's an accident waiting to happen.
| mkitclr wrote: |
| Is there a utility to handle that huge volumes? |
The 2 you've mentioned will do this. Could you be a bit more specific than just "failing on huge volumes"? Like any error messages, codes, that sort of thing?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mkitclr |
| Posted: Fri May 31, 2013 11:39 am Post subject: Volume testing |
|
|
Novice
Joined: 19 May 2011
Posts: 16
|
Vitor-Thanks for the reply
| Quote: |
| This is a very, very, very bad idea. One trivial configuration error and you'll find WMQ's name resolution sending pre-production data to your live system. |
Same name across all Envs is the practice happening at my client
| Quote: |
| The 2 you've mentioned will do this. Could you be a bit more specific than just "failing on huge volumes"? Like any error messages, codes, that sort of thing? |
./q -m XXXXXX -o YYYYYYY -L 100000 -f /usr/local/ZZZZZZZ
MQSeries Q Program by Paul Clarke [ V6.0.0 Build:Jun 12 2012 ]
Can not open input file '/usr/local/ZZZ'
file size 4842491013 bytes. |
|
| Back to top |
|
|
| mkitclr |
| Posted: Fri May 31, 2013 11:41 am Post subject: Volume testing |
|
|
Novice
Joined: 19 May 2011
Posts: 16
|
file is there on the location..
ls -l /usr/local/ZZZZZ
-rw-r--r-- 1 id grp 4842491013 May 23 13:24 /usr/local/ZZZZZZ |
|
| Back to top |
|
|
| mkitclr |
| Posted: Fri May 31, 2013 11:44 am Post subject: Volume testing |
|
|
Novice
Joined: 19 May 2011
Posts: 16
|
One more interesting thing...
I provided the Remote Queue Definition with the Remote Queue Manager attribute as
RQMNAME.Remotehostname ...message landed in the DLQ of Remote QMGR
RQMNAMERemotehostname ....message landed in the DLQ of Remote QMGR
how come it came to know to land in the remote side if the RQMGR I provided is not there or is there any format (string) where I can provide the RQMGR name associated with the hostname. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri May 31, 2013 11:45 am Post subject: Re: Volume testing |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mkitclr wrote: |
| Same name across all Envs is the practice happening at my client |
This does not alter the fact it is a very very bad practice. The same queue names across all environments? Yes, though others here will post differing views any moment. The same queue manager name? No.
At best, it means your client can never use a WMQ cluster to load balance and scale. At best. At worst data from one environment will escape into another.
| mkitclr wrote: |
./q -m XXXXXX -o YYYYYYY -L 100000 -f /usr/local/ZZZZZZZ
MQSeries Q Program by Paul Clarke [ V6.0.0 Build:Jun 12 2012 ]
Can not open input file '/usr/local/ZZZ'
file size 4842491013 bytes. |
And the id opening the file has the permissions needed to open it? The OS supports files of that size? The file name really is ZZZ not zzz or Zzz? etc? etc?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri May 31, 2013 11:48 am Post subject: Re: Volume testing |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mkitclr wrote: |
| how come it came to know to land in the remote side if the RQMGR I provided is not there or is there any format (string) where I can provide the RQMGR name associated with the hostname. |
WMQ only knows about hostnames from your channel configuration. If you've forced (as you must have) the message down the channel that points to the other queue manager then it will end up there, at which point WMQ will attempt name resolution, fail, and DLQ the message.
Your WMQ topology is at best against best practice. Having identically named queue managers in the same estate leads to this and other problems.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mkitclr |
| Posted: Fri May 31, 2013 11:48 am Post subject: Volume testing |
|
|
Novice
Joined: 19 May 2011
Posts: 16
|
| Quote: |
| And the id opening the file has the permissions needed to open it? The OS supports files of that size? The file name really is ZZZ not zzz or Zzz? etc? etc? |
Id has full access to the file, OS supports-Yes,,sorry for the file name confusion it is consistent, but not ZZZZZ(thats just a typr)[/quote] |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri May 31, 2013 11:50 am Post subject: Re: Volume testing |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mkitclr wrote: |
| Id has full access to the file, OS supports-Yes,,sorry for the file name confusion it is consistent, but not ZZZZZ(thats just a typr) |
Which Unix is this? Does the file open using the same id you're running q with if you use cat?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mkitclr |
| Posted: Fri May 31, 2013 12:02 pm Post subject: Volume testing |
|
|
Novice
Joined: 19 May 2011
Posts: 16
|
| Quote: |
| Which Unix is this? Does the file open using the same id you're running q with if you use cat? |
it is RHEL..2.6.18-308.1.1.el5 #1 SMP Fri Feb 17 16:51:01 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
Yes..I could open it with all kind of browse and edit utilities(vi,head,tail,cat)
| Quote: |
| Your WMQ topology is at best against best practice |
I know..here I am trying diff ways to aceive it.
| Quote: |
| Having identically named queue managers in the same estate leads to this and other problems |
I Agree..will push to change. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri May 31, 2013 12:07 pm Post subject: Re: Volume testing |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mkitclr wrote: |
| Yes..I could open it with all kind of browse and edit utilities(vi,head,tail,cat) |
Well I've never enountered a file limit in q but I've never used it on that OS.
Not sure what else you could use that wouldn't encounter the same problem. I'm sure someone else will have a suggestion.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mkitclr |
| Posted: Fri May 31, 2013 12:18 pm Post subject: Volume testing |
|
|
Novice
Joined: 19 May 2011
Posts: 16
|
Thanks to my SYS admin..
He says the 32 bit version of the programs will have trouble handling the files that are larger than 2gb..here q utility is 32 bit..and I couldn't find any 64 bit utility. |
|
| Back to top |
|
|
| Cressida |
| Posted: Sat Jun 01, 2013 12:13 am Post subject: |
|
|
Disciple
Joined: 13 Jul 2007
Posts: 157
|
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Jun 01, 2013 11:02 am Post subject: Re: Volume testing |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mkitclr wrote: |
Thanks to my SYS admin..
He says the 32 bit version of the programs will have trouble handling the files that are larger than 2gb..here q utility is 32 bit..and I couldn't find any 64 bit utility. |
Looks like an OS limitation to me. Have you tried recompiling q with 64 bit binaries? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mvic |
| Posted: Sat Jun 01, 2013 1:06 pm Post subject: Re: Volume testing |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| mkitclr wrote: |
| file size 4842491013 bytes. |
Why is your test trying to work with such a large file? Is this representative of the type of thing that will be done in your prod env? |
|
| Back to top |
|
|
|
|
