| Author |
Message
|
| rrampuria |
 Posted: Thu Jul 25, 2019 8:28 am Post subject: IBM MQ Client application reporting AMQ9211 |
 |
|
Newbie
Joined: 25 Jul 2019
Posts: 7
|
Hi All,
I am running MQClient rpm in rhel6 docker image.
The docker container is running on ubuntu 16 vm
MQClient version : 8.0.0.0-12
MQClient application is reporting:
"MQSeries error: MQCONNX returned code 2071 (MQRC_STORAGE_NOT_AVAILABLE)"
Then I checked the mqclient logs in /var/mqm/errors/ folder
AMQ9211: Error allocating storage.
EXPLANATION:
The program was unable to obtain enough storage.
ACTION:
Stop some programs which are using storage and retry the operation. If the
problem persists contact your systems administrator.
----- amqccisa.c : 5577 -------------------------------------------------------
07/24/19 13:53:33 - Process(1181.1) User(admin) Program(FKRUN)
Host(dcg14compfircofltrserv12) Installation(Installation1)
VRMF(7.5.0.2) QMgr(.)
AMQ9211: Error allocating storage.
EXPLANATION:
The program was unable to obtain enough storage.
ACTION:
Stop some programs which are using storage and retry the operation. If the
problem persists contact your systems administrator.
I checked with my system administrator, according to them the vm has enough space free.
Can anyone help me to figure this out. If i can know which location in the vm its referring to... then I can check again with my sys admin team.
Please let me know if you require more logs or information. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Jul 25, 2019 9:55 am Post subject: Re: IBM MQ Client application reporting AMQ9211 |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rrampuria wrote: |
| I checked with my system administrator, according to them the vm has enough space free. |
Enough space where? There might very well be 2Gb free in /opt but that's no use to something writing to /var or /tmp
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jul 25, 2019 10:16 am Post subject: Re: IBM MQ Client application reporting AMQ9211 |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Vitor wrote: |
| rrampuria wrote: |
| I checked with my system administrator, according to them the vm has enough space free. |
Enough space where? There might very well be 2Gb free in /opt but that's no use to something writing to /var or /tmp |
According to them, exactly how big is the vm's file system? How much space is used? Free? "enough" is not precise.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| rrampuria |
| Posted: Thu Jul 25, 2019 10:58 am Post subject: |
|
|
Newbie
Joined: 25 Jul 2019
Posts: 7
|
[admin@dcg14compfircofltrserv2 compfircoscanserv]$ free -h
total used free shared buffers cached
Mem: 503G 178G 325G 149M 6.2M 70G
-/+ buffers/cache: 107G 396G
Swap: 8.0G 0B 8.0G
[admin@dcg14compfircofltrserv2 compfircoscanserv]$ df -h
Filesystem Size Used Avail Use% Mounted on
none 3.5T 71G 3.4T 2% /
tmpfs 252G 0 252G 0% /dev
tmpfs 252G 0 252G 0% /sys/fs/cgroup
/dev/sda6 3.5T 71G 3.4T 2% /etc/hostname
/dev/sda6 3.5T 71G 3.4T 2% /etc/hosts
shm 64M 0 64M 0% /dev/shm
/dev/sda4 10G 2.6G 7.5G 26% /etc/resolv.conf
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/ibm_mq_ssl
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/customconfig
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/RequesterGreen/log
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/RequesterBlue/log
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/Proxy/log/Proxy
tmpfs 252G 0 252G 0% /proc/acpi
tmpfs 252G 0 252G 0% /proc/kcore
tmpfs 252G 0 252G 0% /proc/keys
tmpfs 252G 0 252G 0% /proc/timer_list
tmpfs 252G 0 252G 0% /proc/sched_debug
tmpfs 252G 0 252G 0% /sys/firmware
tmpfs 252G 0 252G 0% /proc/scsi
Any other location you want me to check? |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jul 25, 2019 11:25 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rrampuria wrote: |
| Any other location you want me to check? |
Wherever you have /var mounted.
Additionally, what is the Docker policy controlling storage access & limits? Especially for non-specific (root) mounts?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Jul 25, 2019 3:32 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
Try running MQ trace, it will show where it is failing. Storage can also relate to memory.
BTW, /var/mqm should be a separate file system, otherwise MQ client dumps (FDC / FFST files) could fill up the /var or root file system, with disasterous consequences.
_________________
Glenn |
|
| Back to top |
|
|
| rrampuria |
| Posted: Wed Jul 31, 2019 11:07 am Post subject: |
|
|
Newbie
Joined: 25 Jul 2019
Posts: 7
|
| how should i run mqtrace in client side? or can i increase the log level to know which storage its talking about? |
|
| Back to top |
|
|
| rrampuria |
| Posted: Wed Jul 31, 2019 2:23 pm Post subject: |
|
|
Newbie
Joined: 25 Jul 2019
Posts: 7
|
Also does the mqclient uses the shared memory or primary memory?
| gbaddeley wrote: |
Try running MQ trace, it will show where it is failing. Storage can also relate to memory.
BTW, /var/mqm should be a separate file system, otherwise MQ client dumps (FDC / FFST files) could fill up the /var or root file system, with disasterous consequences. |
|
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Jul 31, 2019 3:30 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| rrampuria wrote: |
| how should i run mqtrace in client side? or can i increase the log level to know which storage its talking about? |
Yes, try running trace on the client. I suspect the storage issue is on the qmgr side. It might also be worth trying a MQ client sample program (eg. amqsputc) on the qmgr host using the same svrconn channel config and queue name.
_________________
Glenn |
|
| Back to top |
|
|
| rrampuria |
| Posted: Thu Aug 01, 2019 6:24 am Post subject: |
|
|
Newbie
Joined: 25 Jul 2019
Posts: 7
|
Can you please tell me how to run the trace on client side?
What should i add in mqclient.ini file to enable it?
Also .. you said yes for shared memory or primary memory?
| gbaddeley wrote: |
| rrampuria wrote: |
| how should i run mqtrace in client side? or can i increase the log level to know which storage its talking about? |
Yes, try running trace on the client. I suspect the storage issue is on the qmgr side. It might also be worth trying a MQ client sample program (eg. amqsputc) on the qmgr host using the same svrconn channel config and queue name. |
|
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Aug 01, 2019 8:15 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Found this MQ v7.5 closed APAR and fixpacks searching google for 'amqccisa+storage'
http://www-01.ibm.com/support/docview.wss?uid=swg1IC88852
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| rrampuria |
| Posted: Thu Aug 01, 2019 12:44 pm Post subject: |
|
|
Newbie
Joined: 25 Jul 2019
Posts: 7
|
Last I tried with version 8.0.0.0-12 too... same error
|
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Aug 01, 2019 1:22 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| rrampuria wrote: |
Last I tried with version 8.0.0.0-12 too... same error
|
8.0.0.0 of what? Client or Server?
What client version? Errors in client error logs? (For the same test)
What server version? Errors in server error logs? (For the same test)
Did you try amqsputc from the client platform as suggested? What were the results?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| rrampuria |
| Posted: Fri Aug 02, 2019 9:30 am Post subject: |
|
|
Newbie
Joined: 25 Jul 2019
Posts: 7
|
Client side: version 8
server side: 9.0.4
In production setup, we cant run amqsputc commands...
| bruce2359 wrote: |
| rrampuria wrote: |
Last I tried with version 8.0.0.0-12 too... same error
|
8.0.0.0 of what? Client or Server?
What client version? Errors in client error logs? (For the same test)
What server version? Errors in server error logs? (For the same test)
Did you try amqsputc from the client platform as suggested? What were the results? |
|
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Aug 02, 2019 9:54 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
If you can't run IBM's supplied client samples in prod, do you have a development environment where you can replicate this issue?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
