| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Efficient way to delete *ALL* messages from a Queue. |
View previous topic :: View next topic |
| Author |
Message
|
| DPatrick |
 Posted: Mon Jun 07, 2004 7:52 am Post subject: Efficient way to delete *ALL* messages from a Queue. |
 |
|
Newbie
Joined: 07 Jun 2004
Posts: 2
|
I use C++ and the 5.3 MQ C client API.
I have a queue that at the end of the day can have around 10,000 messages totaling about 40 megabytes .
Currently to do a delete, I do a non-browsing MQGET with a zero length buffer and discard the message
It is taking about 20 minutes to delete all the messages
Ideally I'd like an API call that would allow me to tell the queue manager to delete all the messages on the queue, but I havent been able to find one. if thats not available is there any advice/tips for speeding up the processing of a deletetion ?
( below is relevant snippet of my code )
| Code: |
//-----------------------------------------------------------------------
// Setup the Get Message Options structure
//-----------------------------------------------------------------------
MQGMO GetMessageOptions = { MQGMO_DEFAULT };
GetMessageOptions.MatchOptions = MQMO_MATCH_MSG_ID
| MQMO_MATCH_CORREL_ID;
GetMessageOptions.Options = MQGMO_NO_WAIT
| MQGMO_ACCEPT_TRUNCATED_MSG
| MQGMO_FAIL_IF_QUIESCING;
//-----------------------------------------------------------------------
// Retrieve the message from the queue
//-----------------------------------------------------------------------
long DataLength;
MQGET( m_Data->m_TheQueue->QData()->m_Connection
, m_Data->m_BrowseCursor
, &newMessageDescriptor
, &GetMessageOptions
, 0 // buffersize
, 0 // buffer
, &DataLength
, &m_Data->m_lastCompCode
, &m_Data->m_lastReason
); |
|
|
| Back to top |
|
 |
| clindsey |
| Posted: Mon Jun 07, 2004 9:20 am Post subject: |
|
|
Knight
Joined: 12 Jul 2002
Posts: 586
Location: Dallas, Tx
|
Give this a try:
| Code: |
/******************************************************************************/
/* */
/* Program name: CLEARQ.C */
/* */
/* Description: C program to clear all messages from a queue using */
/* the MQSeries Administration Interface (MQAI). */
/******************************************************************************/
/* */
/* Function: */
/* Clear all messages from a queue using the MQAI interface */
/* */
/* - A PCF command is built from items placed into an MQAI administration */
/* bag. */
/* These are:- */
/* - The name argv[1] */
/* */
/* - The mqExecute call is executed with the command MQCMD_CLEAR_Q. */
/* The call generates the correct PCF structure. */
/* The default options to the call are used so that the command is sent */
/* to the SYSTEM.ADMIN.COMMAND.QUEUE. */
/* The reply from the command server is placed on a temporary dynamic */
/* queue. */
/* The reply from the MQCMD_CLEAR_Q command is read from the */
/* temporary queue and formatted into the response bag. */
/* */
/* - The completion code from the mqExecute call is checked and if there */
/* is a failure from the command server, then the code returned by the */
/* command server is retrieved from the system bag that has been */
/* embedded in the response bag to the mqExecute call. */
/* */
/* - If the call is successful, there are no attributes returned for */
/* MQCMD_CLEAR_Q other than complettion code information. */
/* */
/* Note: The command server must be running. */
/* */
/******************************************************************************/
/* */
/* clearq has 2 parameters: */
/* queue name (required) */
/* queue manager name (optional) */
/* */
/******************************************************************************/
/******************************************************************************/
/* Includes */
/******************************************************************************/
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <ctype.h>
#include <cmqc.h> /* MQI */
#include <cmqcfc.h> /* PCF */
#include <cmqbc.h> /* MQAI */
/******************************************************************************/
/* Function prototypes */
/******************************************************************************/
void CheckCallResult(MQCHAR *, MQLONG , MQLONG);
/******************************************************************************/
/* Function: main */
/******************************************************************************/
int main(int argc, char *argv[])
{
/***************************************************************************/
/* MQAI variables */
/***************************************************************************/
MQHCONN hConn; /* handle to MQ connection */
MQCHAR qmName[MQ_Q_MGR_NAME_LENGTH+1]=""; /* default QMgr name */
MQLONG reason; /* reason code */
MQLONG connReason; /* MQCONN reason code */
MQLONG compCode; /* completion code */
MQHBAG adminBag = MQHB_UNUSABLE_HBAG; /* admin bag for mqExecute */
MQHBAG responseBag = MQHB_UNUSABLE_HBAG;/* response bag for mqExecute */
MQHBAG errorBag; /* bag containing cmd server error */
MQLONG mqExecuteCC; /* mqExecute completion code */
MQLONG mqExecuteRC; /* mqExecute reason code */
/***************************************************************************/
/* Connect to the queue manager */
/***************************************************************************/
if (argc > 2)
strncpy(qmName, argv[2], (size_t)MQ_Q_MGR_NAME_LENGTH);
MQCONN(qmName, &hConn, &compCode, &connReason);
/***************************************************************************/
/* Report the reason and stop if the connection failed. */
/***************************************************************************/
if (compCode == MQCC_FAILED)
{
CheckCallResult("Queue Manager connection", compCode, connReason);
exit( (int)connReason);
}
/***************************************************************************/
/* Create an admin bag for the mqExecute call */
/***************************************************************************/
mqCreateBag(MQCBO_ADMIN_BAG, &adminBag, &compCode, &reason);
CheckCallResult("Create admin bag", compCode, reason);
/***************************************************************************/
/* Create a response bag for the mqExecute call */
/***************************************************************************/
mqCreateBag(MQCBO_ADMIN_BAG, &responseBag, &compCode, &reason);
CheckCallResult("Create response bag", compCode, reason);
/***************************************************************************/
/* Put the queue name into the admin bag */
/***************************************************************************/
mqAddString(adminBag, MQCA_Q_NAME, MQBL_NULL_TERMINATED, argv[1], &compCode, &reason);
CheckCallResult("Add q name", compCode, reason);
/***************************************************************************/
/* Send the command to clear the local queue. */
/* The mqExecute call creates the PCF structure required, sends it to */
/* the command server, and receives the reply from the command server into */
/* the response bag. */
/***************************************************************************/
mqExecute(hConn, /* MQ connection handle */
MQCMD_CLEAR_Q, /* Command to be executed */
MQHB_NONE, /* No options bag */
adminBag, /* Handle to bag containing commands */
responseBag, /* Handle to bag to receive the response*/
MQHO_NONE, /* Put msg on SYSTEM.ADMIN.COMMAND.QUEUE*/
MQHO_NONE, /* Create a dynamic q for the response */
&compCode, /* Completion code from the mqexecute */
&reason); /* Reason code from mqexecute call */
/***************************************************************************/
/* Check the command server is started. If not exit. */
/***************************************************************************/
if (reason == MQRC_CMD_SERVER_NOT_AVAILABLE)
{
printf("Please start the command server: <strmqcsv QMgrName>\n");
MQDISC(&hConn, &compCode, &reason);
CheckCallResult("Disconnect from Queue Manager", compCode, reason);
exit(98);
}
/***************************************************************************/
/* Check the result from mqExecute call. If successful find the current */
/* depths of all the local queues. If failed find the error. */
/***************************************************************************/
switch (reason)
{
case MQCC_OK:
printf("Queue %s successfully cleared\n", argv[1]);
break;
case MQRCCF_COMMAND_FAILED:
/*************************************************************************/
/* If the command fails get the system bag handle out of the mqexecute */
/* response bag.This bag contains the reason from the command server */
/* why the command failed. */
/*************************************************************************/
mqInquireBag(responseBag, MQHA_BAG_HANDLE, 0, &errorBag, &compCode, &reason);
CheckCallResult("Get the result bag handle", compCode, reason);
/***********************************************************************/
/* Get the completion code and reason code, returned by the command */
/* server, from the embedded error bag. */
/***********************************************************************/
mqInquireInteger(errorBag, MQIASY_COMP_CODE, MQIND_NONE, &mqExecuteCC,
&compCode, &reason );
CheckCallResult("Get the completion code from the result bag", compCode, reason);
mqInquireInteger(errorBag, MQIASY_REASON, MQIND_NONE, &mqExecuteRC,
&compCode, &reason);
CheckCallResult("Get the reason code from the result bag", compCode, reason);
if (mqExecuteRC == MQRC_Q_NOT_EMPTY)
{
printf("Queue contains uncommitted messages and cannot be cleared.\n");
}
else
{
printf("Error returned by the command server: Completion Code = %ld : Reason = %ld\n",
mqExecuteCC, mqExecuteRC);
}
break;
default:
printf("PCF command failed: Completion Code = %ld : Reason = %ld\n",
compCode, reason);
break;
}
/***************************************************************************/
/* Delete the admin bag if successfully created. */
/***************************************************************************/
if (adminBag != MQHB_UNUSABLE_HBAG)
{
mqDeleteBag(&adminBag, &compCode, &reason);
CheckCallResult("Delete the admin bag", compCode, reason);
}
/***************************************************************************/
/* Delete the response bag if successfully created. */
/***************************************************************************/
if (responseBag != MQHB_UNUSABLE_HBAG)
{
mqDeleteBag(&responseBag, &compCode, &reason);
CheckCallResult("Delete the response bag", compCode, reason);
}
/***************************************************************************/
/* Disconnect from the queue manager if not already connected */
/***************************************************************************/
if (connReason != MQRC_ALREADY_CONNECTED)
{
MQDISC(&hConn, &compCode, &reason);
CheckCallResult("Disconnect from Queue Manager", compCode, reason);
}
return 0;
}
/******************************************************************************/
/* */
/* Function: CheckCallResult */
/* */
/******************************************************************************/
/* */
/* Input Parameters: Description of call */
/* Completion code */
/* Reason code */
/* */
/* Output Parameters: None */
/* */
/* Logic: Display the description of the call, the completion code and the */
/* reason code if the completion code is not successful */
/* */
/******************************************************************************/
void CheckCallResult(char *callText, MQLONG cc, MQLONG rc)
{
if (cc != MQCC_OK)
printf("%s failed: Completion Code = %ld : Reason = %ld\n", callText, cc, rc);
}
|
It works well for a large number of messages on the queue. For a small q depth, I have another pgm that I use that is a modification of amqsget0.c. It sets the buffer size to 1 bytes and then accepts truncated messages.
Hope this helps.
Charlie |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Mon Jun 07, 2004 11:29 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
is the queue always "in use"?
otherwise CLEAR QL(...) would be far more efficient.
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| DPatrick |
| Posted: Mon Jun 07, 2004 12:04 pm Post subject: |
|
|
Newbie
Joined: 07 Jun 2004
Posts: 2
|
First .. thanks to clindsey .. the code snippet was almost exactly what I was looking for. I tested it out and it worked perfectly.
The problem was, as MichaelDag, caught on ... the code doesnt work unless nobody else has the queue open .. and unfortunately this will not be the case .. so I can't use it  |
|
| Back to top |
|
|
| bduncan |
| Posted: Mon Jun 07, 2004 5:56 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
What about temporarily put/get disabling the queue to get those pesky applications to give up their handles? Of course this would require that those applications were coded in such a way to handle this condition gracefully, and then retry a few seconds later, presumeably by which time your clearq program will be done running.
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
| Venku |
| Posted: Mon Jun 07, 2004 9:37 pm Post subject: ClearQ using an MQSeries utility. |
|
|
Novice
Joined: 02 Jun 2004
Posts: 10
Location: Newyork
|
We use an IBM Supplied utility program CSQUTIL for deleting the messages from the queue. We will provide the queue name and a parameter called “Empty” to clear the queue. We wrote a program too but the deleting the message from the queue using the utility is more faster and efficient. This utility is available on Z/OS, OS/390 and MVS. Not sure if it is available as a part of Web MQSeries utilities on other platforms.
http://publibfp.boulder.ibm.com/epubs/html/csqsaw02/csqsaw02tfrm.htm
Best wishes Venkat
 |
|
| Back to top |
|
|
| kevinf2349 |
| Posted: Tue Jun 08, 2004 4:58 am Post subject: |
|
|
Grand Master
Joined: 28 Feb 2003
Posts: 1311
Location: USA
|
| I have a little utility from MQ Solutions that works just fine even if the queue is open. It works on Windows but can client to queue managers. If you want I will check with them about providing you with a copy. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jun 08, 2004 5:29 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| DPatrick wrote: |
First .. thanks to clindsey .. the code snippet was almost exactly what I was looking for. I tested it out and it worked perfectly.
The problem was, as MichaelDag, caught on ... the code doesnt work unless nobody else has the queue open .. and unfortunately this will not be the case .. so I can't use it |
I'm a little confused about this. The code clindsey posted appears to generate a PCF message that executes 'CLEAR QUEUE'.
Why would it work this way when it doesn't work from an MQSC command (which just generates a PCF message, as I understand it)?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| clindsey |
| Posted: Tue Jun 08, 2004 6:42 am Post subject: |
|
|
Knight
Joined: 12 Jul 2002
Posts: 586
Location: Dallas, Tx
|
Jeff,
I posted the code because I thought he may be looking for api calls that a client could call. You are absolutely correct that it makes a CLEAR QUEUE call.
The fact is that neither method of issuing CLEAR QUEUE, PCF nor RUNMQSC will work if a process has the queue open. AFAIK, if you can't stop other processes first, then you have to use the method he was using which is to get each message (preferable truncated) one at a time.
Charlie |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Tue Jun 08, 2004 12:32 pm Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| jefflowrey wrote: |
... from an MQSC command (which just generates a PCF message, as I understand it)? |
just to clarify, as I understand it PCF messages are consumed by the command server and executed.
MQSC (or runmqsc) can be used without the command server running, so uses a different mechanism to "talk" to the QueueManager...
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| mqonnet |
| Posted: Wed Jun 09, 2004 5:58 am Post subject: |
|
|
Grand Master
Joined: 18 Feb 2002
Posts: 1114
Location: Boston, Ma, Usa.
|
"jefflowrey wrote:
... from an MQSC command (which just generates a PCF message, as I understand it)?
just to clarify, as I understand it PCF messages are consumed by the command server and executed.
MQSC (or runmqsc) can be used without the command server running, so uses a different mechanism to "talk" to the QueueManager..."
---Michael, IMHO, you and jeff both are right in a way. Rumqsc does generate PCF messages but i believe it is not executed by the command server as is the case when you send a pcf message from an application.
You can prove this yourself. Run a trace individually with runmqsc, a pcf application and a regular application. You would see something like PCMINQUIRE which is common to runmqsc and pcf application but NOT to a regular application(making mq api calls, say mqget, put).
Cheers
Kumar
_________________
IBM Certified WebSphere MQ V5.3 Developer
IBM Certified WebSphere MQ V5.3 Solution Designer
IBM Certified WebSphere MQ V5.3 System Administrator |
|
| Back to top |
|
|
| markt |
| Posted: Wed Jun 09, 2004 6:06 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
runmqsc does NOT generate PCF messages.
Both runmqsc and the command server take their independent input formats and parse them into a common set of internal function calls, and then do the reverse for mapping to response formats. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jun 09, 2004 6:07 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| markt wrote: |
| runmqsc does NOT generate PCF messages. |
That's my new fact for the day.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Wed Jun 09, 2004 6:32 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| markt wrote: |
runmqsc does NOT generate PCF messages.
Both runmqsc and the command server take their independent input formats and parse them into a common set of internal function calls, and then do the reverse for mapping to response formats. |
That is exactly what a little birdie (who shall remain nameless) told me once...
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| mqonnet |
| Posted: Wed Jun 09, 2004 7:43 am Post subject: |
|
|
Grand Master
Joined: 18 Feb 2002
Posts: 1114
Location: Boston, Ma, Usa.
|
"Both runmqsc and the command server take their independent input formats and parse them into a common set of internal function calls, "
---You are right here. When i said earlier that both do generate PCF messages, i meant, both do take the same route as in my example earlier of PCMINQUIRE.
Be it a pcf application or be it runmqsc, they both call the same routines eventually. What parameters are sent to these calls, distinguishes one from the other in the form of PCF messages.
Cheers
Kumar
_________________
IBM Certified WebSphere MQ V5.3 Developer
IBM Certified WebSphere MQ V5.3 Solution Designer
IBM Certified WebSphere MQ V5.3 System Administrator |
|
| Back to top |
|
|
|
|
|
|
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
