| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Reporting the depth of a queue (& other interesting stuf |
« View previous topic :: View next topic » |
| Author |
Message
|
| jcalderone |
 Posted: Mon Nov 02, 2009 9:17 am Post subject: Reporting the depth of a queue (& other interesting stuf |
 |
|
Novice
Joined: 02 Nov 2009
Posts: 12
|
Hello,
I'm working on extending an existing monitoring tool with support for monitoring various aspects of MQ queues and queue managers. The first task I'm tackling is trying to report on the current queue depth of a specified queue on a specified queue manager.
I see this question has come up a lot in this forum, but apparently mostly from people who want this information for the wrong reason (ie, to know how many messages to get from the queue). My only interest in this information is to report it to the monitoring application so a data point can be added to a pretty graph. 
Reading through the IBM-provided API documentation, I've found a few possible approaches to this task. The first thing I found was the MQINQ API. It seems I should be able to use this with the MQIA_CURRENT_Q_DEPTH integer selector to find the instantaneous physical message depth for a particular local queue. However, I've had trouble putting this into practice. I'm working in C and I've based much of my work thus far off of the examples provided with the MQ client library.
My best attempt so far results in an error with reason 2068 - MQRC_SELECTOR_NOT_FOR_TYPE. I'm somewhat stuck at this point, since the documentation seems to suggest that MQIA_CURRENT_Q_DEPTH is perfectly valid for local queue objects.
The following is my attempt at a minimal example demonstrating the problem I'm having. It includes a few things which I don't think should be necessary to get the current queue depth but which I have tried because of messages I found on this forum or elsewhere. I am most interested in why this fails to retrieve the depth of the specified queue, but any other suggestions about MQ API usage would be appreciated as well; this is my first use of MQ in any capacity.
| Code: |
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <limits.h>
#include <assert.h>
#include <cmqc.h>
#include <cmqbc.h>
#include <cmqxc.h>
typedef char QueueManagerName_t[MQ_Q_MGR_NAME_LENGTH];
typedef char QueueName_t[MQ_Q_NAME_LENGTH];
typedef struct MQResult {
MQLONG completionCode;
MQLONG reason;
} MQResult;
/**
* Report the given failure reason on stderr and exit the process with the given exit code.
*/
void fail(MQResult *error, const char* operation) {
fprintf(
stdout,
"%s\ncompletion code: %d\nreason: %d\n",
operation, error->completionCode, error->reason);
exit(-1);
}
/**
* Check the given MQResult for non-success and fail if found.
*/
void check(MQResult *error, const char *operation) {
if (error->completionCode != MQCC_OK) {
fail(error, operation);
}
}
/**
* Write the given statistic to stdout.
*/
void reportStatistic(const char* stat, int value) {
printf("%s: %s\n", stat, value);
}
/**
* Open and return a new connection to the named queue manager. Call closeConnection
* to clean up the connection.
*
* @param queueManagerName: The name of the queue manager to which to connect.
*
* @param error: A pointer to an MQResult structure which will be populated with
* any error information if an internal MQ call fails. The contents
* may be overwritten even if no errors occur.
*
*/
MQHCONN openConnection(QueueManagerName_t queueManagerName,
const char *host, int port, const char *channel, MQResult *error) {
MQHCONN connection;
MQCNO connectOptions = {MQCNO_DEFAULT};
MQCD clientConnection = {MQCD_CLIENT_CONN_DEFAULT};
connectOptions.Version = MQCNO_VERSION_2;
/* XXX snprintf here would make this actually safe, but Windows doesn't have snprintf
* and I don't have a non-Windows development environment.
*/
sprintf(clientConnection.ConnectionName, "%s(%d)", host, port);
clientConnection.TransportType = MQXPT_TCP;
strncpy(clientConnection.ChannelName, channel, MQ_CHANNEL_NAME_LENGTH);
connectOptions.ClientConnPtr = &clientConnection;
/* Open a connection to the named queue manager.
*/
MQCONNX(
queueManagerName,
&connectOptions,
&connection,
&error->completionCode,
&error->reason);
/* Fail if it could not be opened.
*/
if (error->completionCode!= MQCC_OK) {
return 0;
}
return connection;
}
/**
* Close a connection to a queue manager previously opened with openConnection.
*/
int closeConnection(MQHCONN connection, MQResult *error) {
MQDISC(
&connection,
&error->completionCode,
&error->reason);
/* Fail if something went wrong during disconnection.
*/
if (error->completionCode != MQCC_OK) {
return -1;
}
return 0;
}
int getQueueAttributeFromHandle(MQHCONN connection, MQHOBJ inquireObject, MQLONG attribute, MQResult *error) {
/* The only thing we're going to inquire about is the current queue depth.
*/
MQLONG selectorsTable[1] = {attribute};
/* MQINQ needs to know how many elements are in selectorsTable. Specify
* that here.
*/
MQLONG selectorCount = 1;
/* It also needs to know how many of each type of value is being requested.
* This function will only support integer attributes for now.
*/
MQLONG intAttrCount = 1;
/* Allocate space for the result.
*/
MQLONG intAttrsTable[1] = {0};
/* Issue the INQUIRY command.
*/
MQINQ(
connection, inquireObject,
selectorCount, selectorsTable,
intAttrCount, intAttrsTable,
0, NULL,
&error->completionCode,
&error->reason);
/* Otherwise, return the current queue depth - available as the only
* element of the integer attributes out parameter from the MQINQ call.
*/
return intAttrsTable[0];
}
/**
* Retrieve and return an integer attribute of the named queue.
*
* @param connection: An established connection to the queue manager on which
* the queue exists.
*
* @param queueName: The name of the queue on which to operate.
*
* @param attribute: The attribute to retrieve. Only integer attributes are allowed.
*
* @param error: A pointer to an MQResult structure which will be populated with
* any error information if an internal MQ call fails. The contents
* may be overwritten even if no errors occur.
*
* @return: On error, -1. On success, a positive integer giving the value of the
* the indicated queue.
*/
int getQueueAttribute(MQHCONN connection, QueueManagerName_t queueManagerName, QueueName_t queueName, MQLONG attribute, MQResult *error) {
MQLONG value;
MQOD queueDescription = {MQOD_DEFAULT};
/* XXX Try with set in addition to inquire as a hack to force something to be local or something and
* maybe make it work or something. Probably not.
*/
MQLONG openOptions = MQOO_INQUIRE | MQOO_SET | MQOO_RESOLVE_LOCAL_Q;
MQLONG closeOptions = 0;
MQHOBJ inquireObject = 0;
/*
* As an experiment, up the version of the MQOD. Not much of an experiment, since I have no
* control.
*/
queueDescription.Version = MQOD_VERSION_3;
/* Initialize the object description with the name of the queue we're
* interested in.
*/
strncpy(queueDescription.ObjectName, queueName, MQ_OBJECT_NAME_LENGTH);
/* XXX Experiment. Does also setting the queue manager name here make a subsequent MQINQ for
* MQIA_CURRENT_Q_DEPTH work, instead of failing with 2068?
*/
strncpy(queueDescription.ObjectQMgrName, queueManagerName, MQ_Q_MGR_NAME_LENGTH);
/* Clear out the fields we'll inspect later, just to make sure they don't start with garbage
* in them.
*/
memset(queueDescription.ResolvedQName, ' ', MQ_OBJECT_NAME_LENGTH);
memset(queueDescription.ResolvedQMgrName, ' ', MQ_Q_MGR_NAME_LENGTH);
/* Open that queue object for the INQUIRY command.
*/
MQOPEN(
connection,
&queueDescription,
openOptions,
&inquireObject,
&error->completionCode,
&error->reason);
/* If that didn't work, fail now.
*/
if (error->completionCode != MQCC_OK) {
return -1;
}
/*
* Dump some debugging info.
*/
printf(
"ResolvedQName: %s\n"
"ResolvedQMgrName: %s\n"
"RecsPresent: %d\n"
"KnownDestCount: %d\n"
"UnknownDestCount: %d\n"
"InvalidDestCount: %d\n",
queueDescription.ResolvedQName,
queueDescription.ResolvedQMgrName,
queueDescription.RecsPresent,
queueDescription.KnownDestCount,
queueDescription.UnknownDestCount,
queueDescription.InvalidDestCount);
value = getQueueAttributeFromHandle(connection, inquireObject, attribute, error);
/* If the INQUIRY command failed, report that to the caller.
*/
if (error->completionCode != MQCC_OK) {
return -1;
}
/* Close the object handle associated with the queue.
*/
MQCLOSE(
connection, &inquireObject,
closeOptions,
&error->completionCode, &error->reason);
/* And report that failure if it was indeed a failure.
*/
if (error->completionCode!= MQCC_OK) {
return -1;
}
return value;
}
void reportIntegerAttribute(MQHCONN connection, QueueManagerName_t queueManagerName, QueueName_t queueName, const char *description,
MQLONG attribute, MQResult *error) {
int value = getQueueAttribute(connection, queueManagerName, queueName, attribute, error);
/* Negative return value indicates an error of some sort. Let the caller deal with the
* details from the MQResult.
*/
if (value >= 0) {
reportStatistic(description, value);
}
}
/**
* Retrieve and display the number of messages currently on a particular queue.
*
* Five command line arguments are expected:
*
* - The name of the channel to use
* - The host name or IP address of the queue manager to connect to
* - The port number the queue manager is listening on
* - a queue manager name
* - a queue name.
*
* The queue named must be local to the queue manager named.
*/
int main(int argc, char **argv) {
MQHCONN connection;
MQResult error;
int attrCounter, portNumber;
struct {
const char *description;
MQLONG attribute;
const char *failure;
} intAttributes[] = {
{"Current Queue Depth", MQIA_CURRENT_Q_DEPTH, "Failed to determine queue depth."},
};
char *channel, *host, *port, *queueManagerName, *queueName;
/* Display usage information on stderr if invoked with the wrong number of
* parameters.
*/
if (argc != 6) {
fprintf(
stdout,
"Usage: %s <channel name> <host> <port> <queue manager name> <queue name>\n",
argv[0]);
return -1;
}
channel = argv[1];
host = argv[2];
port = argv[3];
queueManagerName = argv[4];
queueName = argv[5];
/* Convert the port to an integer.
*/
portNumber = atoi(port);
if (portNumber == 0 || portNumber == INT_MIN || portNumber == INT_MAX) {
fprintf(stdout, "Invalid port number: %s", port);
return -1;
}
/* Try to connect to the specified queue manager.
*/
connection = openConnection(queueManagerName, host, portNumber, channel, &error);
check(&error, "Failed to open connection.");
/* Try to determine and report a number of integer attributes of the queue.
*/
for (attrCounter = 0; attrCounter < sizeof intAttributes / sizeof intAttributes[0]; ++attrCounter) {
reportIntegerAttribute(
connection, queueManagerName, queueName,
intAttributes[attrCounter].description,
intAttributes[attrCounter].attribute,
&error);
check(&error, intAttributes[attrCounter].failure);
}
/* Clean up the queue manager connection.
*/
closeConnection(connection, &error);
check(&error, "Failed to close connection cleanly.");
return 0;
}
|
I should also mention that I'm building and testing this on Windows. I'm linking against mqic32.lib, which I think is the right library to link against for clients (I understand that the choice of library makes a difference to the behavior of an MQ application, though I don't think I understand yet why one would link against anything other than the client library).
Thanks in advance for any assistance,
Jean-Paul |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Nov 02, 2009 9:36 am Post subject: Re: Reporting the depth of a queue (& other interesting |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jcalderone wrote: |
| My best attempt so far results in an error with reason 2068 - MQRC_SELECTOR_NOT_FOR_TYPE. I'm somewhat stuck at this point, since the documentation seems to suggest that MQIA_CURRENT_Q_DEPTH is perfectly valid for local queue objects. |
It is, and I would suspect that you're getting this error because your code (rightly or due to issues) is trying to inquire against something that isn't a local queue. Note that in WMQ terms this has a very tight definition; while alias and remote queues are indeed "local" to the queue manager they don't have depth attributes.
| jcalderone wrote: |
| I should also mention that I'm building and testing this on Windows. I'm linking against mqic32.lib, which I think is the right library to link against for clients (I understand that the choice of library makes a difference to the behavior of an MQ application, though I don't think I understand yet why one would link against anything other than the client library). |
If the connection method was wrong you'd get a different error code. For the record, you'd use bindings (rather than client) if you're looking for increased reliability and throughput for your application, trading that off against the reduced mobility of the application.
The choice of connection method has no impact on the behavior of an application. An application that uses a client should handle connections that drop, but there's no reason why an application coded that way wouldn't work unmodified relinked as a binding application.
I've also seen a number of applications designed as clients that don't bother with such code. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jcalderone |
| Posted: Wed Nov 04, 2009 8:27 am Post subject: |
|
|
Novice
Joined: 02 Nov 2009
Posts: 12
|
| Quote: |
It is, and I would suspect that you're getting this error because your code (rightly or due to issues) is trying to inquire against something that isn't a local queue.
|
This turns out to be just so. The queue was of type MQQT_REMOTE (your post inspired me to add an MQIA_Q_TYPE lookup before anything else).
I wonder if, finding myself in the position of having a queue that is of type MQQT_REMOTE, there is any way to discover the name of the queue manager to which I need to connect in order to be able to open a corresponding MQQT_LOCAL queue? Earlier I added MQOO_RESOLVE_LOCAL_Q to the open flags I'm using with MQOPEN; from the docs, I thought this would cause the MQOD's ResolvedQMgrName field to be populated with the name of the remote queue manager where the opened queue would be local. Re-reading the docs for this flag, now I think that *omitting* MQOO_RESOLVE_LOCAL_Q will actually trigger this behavior. Is either of these readings correct? If not, is there some other way to learn this information?
Thanks. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Nov 04, 2009 8:33 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| So what happens if the Queue is a QREMOTE on the ResolvedQmgrName too? |
|
| Back to top |
|
|
| jcalderone |
| Posted: Wed Nov 04, 2009 8:58 am Post subject: |
|
|
Novice
Joined: 02 Nov 2009
Posts: 12
|
| mqjeff wrote: |
| So what happens if the Queue is a QREMOTE on the ResolvedQmgrName too? |
I don't know. I guess I would be in trouble? Are you suggesting that's possible? If so, then my idea about using ResolvedQmgrName to find a QLOCAL Queue is broken and I need another idea. Any suggestions? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Nov 04, 2009 9:07 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I'm just wondering how many hops you're willing to make to find the queue depth of a qremote.
And what your business partners might think if you suddenly started monitoring their queues using PCF messages....
Most monitoring apps do nothing to try and resolve remote objects for purposes of monitoring. They give you qdepth of qlocals, and let you manually create connections to other qmgrs. Or do qmgr discovery based on clusters and stuff. But still only let you see depths for QLOCALs. |
|
| Back to top |
|
|
| jcalderone |
| Posted: Wed Nov 04, 2009 9:14 am Post subject: |
|
|
Novice
Joined: 02 Nov 2009
Posts: 12
|
| mqjeff wrote: |
I'm just wondering how many hops you're willing to make to find the queue depth of a qremote.
|
Probably not a good idea to do too many, I suppose? I have little experience here, and no real concept of the cost the operations involved here.
| mqjeff wrote: |
And what your business partners might think if you suddenly started monitoring their queues using PCF messages....
|
I got some pushback about using PCF messages because of the queue creation required. I might be able to overcome these objections if the other solutions end up being worse, of course.
| mqjeff wrote: |
| Most monitoring apps do nothing to try and resolve remote objects for purposes of monitoring. They give you qdepth of qlocals, and let you manually create connections to other qmgrs. Or do qmgr discovery based on clusters and stuff. But still only let you see depths for QLOCALs. |
This is definitely something I'm considering too. I don't think there's much reason to avoid requiring, in the monitoring config, that QLOCALs be specified wherever this information is desired. Having the monitoring tool figure this out automatically would be kind of convenient, but it may not end up being worth the cost. For the moment I'm still investigating all the possibilities and haven't settled on anything yet.
Thanks for your feedback! |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Nov 04, 2009 9:42 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jcalderone wrote: |
| mqjeff wrote: |
I'm just wondering how many hops you're willing to make to find the queue depth of a qremote.
|
Probably not a good idea to do too many, I suppose? I have little experience here, and no real concept of the cost the operations involved here. |
Moving back from the dangers of getting lost in a multi-hop environment, why are you attempting to enquire on remote queue managers? If they're owned by business partners, or even different parts of your business, then your intrusion is likely to be unwelcome and possibly illegal. Also, what will you do with this information? Ring your business partner and tell them their queue manager is broken?
A general rule is that if you're responsible for the queue manager, you install your monitoring solution locally and monitor it there. There's no good reason why such a monitoring agent can't report it's findings to a single, local point; indeed most commerical monitoring solutions do exactly that.
| jcalderone wrote: |
| mqjeff wrote: |
And what your business partners might think if you suddenly started monitoring their queues using PCF messages....
|
I got some pushback about using PCF messages because of the queue creation required. I might be able to overcome these objections if the other solutions end up being worse, of course. |
This sounds a bit like a feel good excuse. I suspect the pushback is more "no way we're accepting PCF commands from there" without sounding like they don't trust you. Remember that PCF commands can do far more than enquire on queue depth and while I am in no way suggesting you are doing anything of the kind, once the door has been opened to allow PCF commands it's hard to close.
It's also a tricky sell to site security, who tend to say "no" shortly after the phrase "receiving remote admin commands" without listening to the rest of the sentance.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Nov 04, 2009 11:10 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9400
Location: US: west coast, almost. Otherwise, enroute.
|
You do understand that this type of monitoring software has already been invented, don't you?
Does your management understand that there is no such thing as free softwre? And that home-grown software must be maintained forever? And forever means that your home-grown software must be analyzed and tested with each new release and upgrade to re-validate that it works?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| jcalderone |
| Posted: Wed Nov 04, 2009 11:27 am Post subject: |
|
|
Novice
Joined: 02 Nov 2009
Posts: 12
|
| bruce2359 wrote: |
You do understand that this type of monitoring software has already been invented, don't you?
|
Maybe, maybe not. I'm greatful for any pointers you have to existing tools that might fit into this situation. I'm very new to WMQ and could easily be ignorant of the perfect existing solution. Since there's already a monitoring system in place, the requirement is to extend that to also be able to monitor certain aspects of an MQ server. An easy way to extend the existing system is with small programs that accept some arguments and report a few numbers in their output.
So, for example, is there an existing tool like this which can report the current depth of a queue? Even better would be something that could report on many or all of the other stats about queues and queue managers (enqueue count, high queue depth, queue empty, etc).
| bruce2359 wrote: |
| Does your management understand that there is no such thing as free softwre? And that home-grown software must be maintained forever? And forever means that your home-grown software must be analyzed and tested with each new release and upgrade to re-validate that it works? |
I'm not certain whether they understand this or not. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Nov 04, 2009 11:36 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
If the existing system is COTS, rather than home-grown, the odds are around 90% that you can spend more money and purchase a set of WMQ extensions from the vendor.
If the existing system is home-grown, then you're at least as well off pushing management to replace the whole shebang with a COTS monitoring solution that also knows MQ as you are cobbling in new functions to monitor something you self-admittedly don't know very well. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Nov 04, 2009 11:42 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9400
Location: US: west coast, almost. Otherwise, enroute.
|
A quick search by Mr. Google for 'mq+monitoring' gave me Results 1 - 10 of about 1,890,000.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Nov 04, 2009 11:45 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9400
Location: US: west coast, almost. Otherwise, enroute.
|
As an exercise: if you inquire on currepth of a queue, and discover that curdepth is 5, what will you do with this information? Or, more accurately, what will those that requested this type of information learn from the fact that curdepth is 5?
What does it means when curdepth is 5?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Last edited by bruce2359 on Wed Nov 04, 2009 11:47 am; edited 1 time in total |
|
| Back to top |
|
|
| jcalderone |
| Posted: Wed Nov 04, 2009 11:46 am Post subject: |
|
|
Novice
Joined: 02 Nov 2009
Posts: 12
|
| It's COTS. It doesn't support WMQ, though. I suspect we could be paying the vendor to implement this functionality rather than doing it ourselves, but that decision was someone else's. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Nov 04, 2009 11:47 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jcalderone wrote: |
| So, for example, is there an existing tool like this which can report the current depth of a queue? Even better would be something that could report on many or all of the other stats about queues and queue managers (enqueue count, high queue depth, queue empty, etc). |
You've just described every commercial monitoring tool with WMQ capabilities! These are (as I'm sure you've discovered) the common things which need to be monitored and possibly acted on.
You could do a lot worse than look through the "Monitoring and tools" section of this forum. Most of the software "usual suspects" have been discussed at one time or another.
Another viable plan is to ask your local admins what they're using to monitor CPU, RAM, disc space and so forth. Most of those monitoring tools have WMQ capability.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
