ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexGeneral IBM MQ SupportIBM MQ Client application reporting AMQ9211

Post new topicReply to topic
IBM MQ Client application reporting AMQ9211 View previous topic :: View next topic
Author Message
rrampuria
PostPosted: Thu Jul 25, 2019 8:28 am Post subject: IBM MQ Client application reporting AMQ9211 Reply with quote

Newbie

Joined: 25 Jul 2019
Posts: 7

Hi All,

I am running MQClient rpm in rhel6 docker image.
The docker container is running on ubuntu 16 vm

MQClient version : 8.0.0.0-12

MQClient application is reporting:

"MQSeries error: MQCONNX returned code 2071 (MQRC_STORAGE_NOT_AVAILABLE)"

Then I checked the mqclient logs in /var/mqm/errors/ folder

AMQ9211: Error allocating storage.
EXPLANATION:
The program was unable to obtain enough storage.
ACTION:
Stop some programs which are using storage and retry the operation. If the
problem persists contact your systems administrator.
----- amqccisa.c : 5577 -------------------------------------------------------
07/24/19 13:53:33 - Process(1181.1) User(admin) Program(FKRUN)
Host(dcg14compfircofltrserv12) Installation(Installation1)
VRMF(7.5.0.2) QMgr(.)
AMQ9211: Error allocating storage.
EXPLANATION:
The program was unable to obtain enough storage.
ACTION:
Stop some programs which are using storage and retry the operation. If the
problem persists contact your systems administrator.


I checked with my system administrator, according to them the vm has enough space free.

Can anyone help me to figure this out. If i can know which location in the vm its referring to... then I can check again with my sys admin team.

Please let me know if you require more logs or information.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Jul 25, 2019 9:55 am Post subject: Re: IBM MQ Client application reporting AMQ9211 Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25823
Location: Ohio, USA

rrampuria wrote:
I checked with my system administrator, according to them the vm has enough space free.


Enough space where? There might very well be 2Gb free in /opt but that's no use to something writing to /var or /tmp
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Jul 25, 2019 10:16 am Post subject: Re: IBM MQ Client application reporting AMQ9211 Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8495
Location: US: west coast, almost. Otherwise, enroute.

Vitor wrote:
rrampuria wrote:
I checked with my system administrator, according to them the vm has enough space free.


Enough space where? There might very well be 2Gb free in /opt but that's no use to something writing to /var or /tmp

According to them, exactly how big is the vm's file system? How much space is used? Free? "enough" is not precise.
_________________
There are two types of people in this world:
1) Those that can extrapolate from incomplete data
Back to top
View user's profile Send private message
rrampuria
PostPosted: Thu Jul 25, 2019 10:58 am Post subject: Reply with quote

Newbie

Joined: 25 Jul 2019
Posts: 7

[admin@dcg14compfircofltrserv2 compfircoscanserv]$ free -h
total used free shared buffers cached
Mem: 503G 178G 325G 149M 6.2M 70G
-/+ buffers/cache: 107G 396G
Swap: 8.0G 0B 8.0G
[admin@dcg14compfircofltrserv2 compfircoscanserv]$ df -h
Filesystem Size Used Avail Use% Mounted on
none 3.5T 71G 3.4T 2% /
tmpfs 252G 0 252G 0% /dev
tmpfs 252G 0 252G 0% /sys/fs/cgroup
/dev/sda6 3.5T 71G 3.4T 2% /etc/hostname
/dev/sda6 3.5T 71G 3.4T 2% /etc/hosts
shm 64M 0 64M 0% /dev/shm
/dev/sda4 10G 2.6G 7.5G 26% /etc/resolv.conf
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/ibm_mq_ssl
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/customconfig
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/RequesterGreen/log
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/RequesterBlue/log
/dev/sda6 3.5T 71G 3.4T 2% /x/web/LIVE/compfircoscanserv/Proxy/log/Proxy
tmpfs 252G 0 252G 0% /proc/acpi
tmpfs 252G 0 252G 0% /proc/kcore
tmpfs 252G 0 252G 0% /proc/keys
tmpfs 252G 0 252G 0% /proc/timer_list
tmpfs 252G 0 252G 0% /proc/sched_debug
tmpfs 252G 0 252G 0% /sys/firmware
tmpfs 252G 0 252G 0% /proc/scsi


Any other location you want me to check?
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Jul 25, 2019 11:25 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25823
Location: Ohio, USA

rrampuria wrote:
Any other location you want me to check?


Wherever you have /var mounted.

Additionally, what is the Docker policy controlling storage access & limits? Especially for non-specific (root) mounts?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Jul 25, 2019 3:32 pm Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2014
Location: Melbourne, Australia

Try running MQ trace, it will show where it is failing. Storage can also relate to memory.

BTW, /var/mqm should be a separate file system, otherwise MQ client dumps (FDC / FFST files) could fill up the /var or root file system, with disasterous consequences.
_________________
Glenn
Back to top
View user's profile Send private message
rrampuria
PostPosted: Wed Jul 31, 2019 11:07 am Post subject: Reply with quote

Newbie

Joined: 25 Jul 2019
Posts: 7

how should i run mqtrace in client side? or can i increase the log level to know which storage its talking about?
Back to top
View user's profile Send private message
rrampuria
PostPosted: Wed Jul 31, 2019 2:23 pm Post subject: Reply with quote

Newbie

Joined: 25 Jul 2019
Posts: 7

Also does the mqclient uses the shared memory or primary memory?

gbaddeley wrote:
Try running MQ trace, it will show where it is failing. Storage can also relate to memory.

BTW, /var/mqm should be a separate file system, otherwise MQ client dumps (FDC / FFST files) could fill up the /var or root file system, with disasterous consequences.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Wed Jul 31, 2019 3:30 pm Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2014
Location: Melbourne, Australia

rrampuria wrote:
how should i run mqtrace in client side? or can i increase the log level to know which storage its talking about?

Yes, try running trace on the client. I suspect the storage issue is on the qmgr side. It might also be worth trying a MQ client sample program (eg. amqsputc) on the qmgr host using the same svrconn channel config and queue name.
_________________
Glenn
Back to top
View user's profile Send private message
rrampuria
PostPosted: Thu Aug 01, 2019 6:24 am Post subject: Reply with quote

Newbie

Joined: 25 Jul 2019
Posts: 7

Can you please tell me how to run the trace on client side?
What should i add in mqclient.ini file to enable it?

Also .. you said yes for shared memory or primary memory?

gbaddeley wrote:
rrampuria wrote:
how should i run mqtrace in client side? or can i increase the log level to know which storage its talking about?

Yes, try running trace on the client. I suspect the storage issue is on the qmgr side. It might also be worth trying a MQ client sample program (eg. amqsputc) on the qmgr host using the same svrconn channel config and queue name.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Aug 01, 2019 8:15 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8495
Location: US: west coast, almost. Otherwise, enroute.

Found this MQ v7.5 closed APAR and fixpacks searching google for 'amqccisa+storage'

http://www-01.ibm.com/support/docview.wss?uid=swg1IC88852
_________________
There are two types of people in this world:
1) Those that can extrapolate from incomplete data
Back to top
View user's profile Send private message
rrampuria
PostPosted: Thu Aug 01, 2019 12:44 pm Post subject: Reply with quote

Newbie

Joined: 25 Jul 2019
Posts: 7

Last I tried with version 8.0.0.0-12 too... same error

bruce2359 wrote:
Found this MQ v7.5 closed APAR and fixpacks searching google for 'amqccisa+storage'

http://www-01.ibm.com/support/docview.wss?uid=swg1IC88852
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Aug 01, 2019 1:22 pm Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8495
Location: US: west coast, almost. Otherwise, enroute.

rrampuria wrote:
Last I tried with version 8.0.0.0-12 too... same error

bruce2359 wrote:
Found this MQ v7.5 closed APAR and fixpacks searching google for 'amqccisa+storage'

http://www-01.ibm.com/support/docview.wss?uid=swg1IC88852

8.0.0.0 of what? Client or Server?

What client version? Errors in client error logs? (For the same test)

What server version? Errors in server error logs? (For the same test)

Did you try amqsputc from the client platform as suggested? What were the results?
_________________
There are two types of people in this world:
1) Those that can extrapolate from incomplete data
Back to top
View user's profile Send private message
rrampuria
PostPosted: Fri Aug 02, 2019 9:30 am Post subject: Reply with quote

Newbie

Joined: 25 Jul 2019
Posts: 7

Client side: version 8
server side: 9.0.4

In production setup, we cant run amqsputc commands...


bruce2359 wrote:
rrampuria wrote:
Last I tried with version 8.0.0.0-12 too... same error

bruce2359 wrote:
Found this MQ v7.5 closed APAR and fixpacks searching google for 'amqccisa+storage'

http://www-01.ibm.com/support/docview.wss?uid=swg1IC88852

8.0.0.0 of what? Client or Server?

What client version? Errors in client error logs? (For the same test)

What server version? Errors in server error logs? (For the same test)

Did you try amqsputc from the client platform as suggested? What were the results?
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Aug 02, 2019 9:54 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8495
Location: US: west coast, almost. Otherwise, enroute.

If you can't run IBM's supplied client samples in prod, do you have a development environment where you can replicate this issue?
_________________
There are two types of people in this world:
1) Those that can extrapolate from incomplete data
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexGeneral IBM MQ SupportIBM MQ Client application reporting AMQ9211
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.