Author |
Message
|
ora |
Posted: Thu Sep 21, 2006 1:59 pm Post subject: Issue w/ concurrent get calls using java class and web ser |
|
|
Newbie
Joined: 20 Sep 2006 Posts: 5
|
Hello,
I have a java class that i used to create a web service. I have one function (putMsgOnRequestQueue) that puts a message on to a request queue and another function that gets message off of a response queue. When I run my script with java (main), I know that I can put and get a successful response. Even when I run a test using a java Client that calls web service, I can put and get a successful response. The issue is when I do multiple calls to web service at a time. Example: 5 concurrent calls. I might get back the first couple of responses then I get null pointer exception (Error 2033 - NO MSG ON QUEUE). When I close queues and disconnect from queue manager, message is on response queue. I have looked into adjusting my wait interval and this problem still occurs. I can see all of my requests on the request queue, so putting message on request queue doesn't seem to be a problem. It is when I attempt to get a message from the response queue. I am not certain of this but, issue seems to be related to using a priority of some sort when responding to request.
FYI: MQ is a clients remote MQ (CICS). Client uses a trigger on the request queue. When I put a message on request queue, a trigger is set so that a reply is put on the response queue.
Java put and get functions below:
/**
* This method puts a message on the Request Queue and returns the correlation
* id.
*
* @param inMsg the message to send.
* @return String the correlation id.
* @exception Exception thrown if an error occurred sending/receiving the message.
*
*/
public String putMsgOnRequestQueue(ina.mq.connector.BaseConnector connect, String inMsg) throws Exception
{
try
{
// Connect to MQ
makeMQConnection( connect );
// Create MQ Message
MQMessage myRequestMsg = new MQMessage();
// create message options
MQPutMessageOptions pmo = new MQPutMessageOptions();
// accept defaults
pmo.options = MQC.MQPMO_NONE;
// Clear the message
myRequestMsg.clearMessage();
// Get next correlation id
getNextCorrelationId();
// Set correlation id
myRequestMsg.correlationId = mqCorrelationId.getBytes();
// Set message id
myRequestMsg.messageId = mqCorrelationId.getBytes();
// Set message format
myRequestMsg.format = MQC.MQFMT_STRING;
// Write the message
myRequestMsg.writeString(inMsg);
// Put message on the queue
mqRequestQueue.put(myRequestMsg, pmo);
finalise();
}
catch (MQException ex)
{
System.out.println("MQ exception occurred : Completion code "
+ ex.completionCode
+ " Reason code "
+ ex.reasonCode);
log.fatal("MQ exception occurred : Completion code "
+ ex.completionCode
+ " Reason code "
+ ex.reasonCode);
//Close all connections
finalise();
// Throw exception
throw ex;
}
catch (java.io.IOException ex)
{
System.out.println("An error occurred reading from message buffer: " + ex);
log.fatal("An error occurred reading from message buffer: " +ex);
//Close all connections
finalise();
// Throw exception
throw ex;
}
// Return correlation id
return mqCorrelationId;
}
/**
* This method retrieves a message from the Response Queue based on the correlation
* id. If the correlation id is null then the next message on the queue is
* retrieved.
*
* @param inCorrelationId the correlation id of the message to retrieve or null
* to retrieve the next message.
* @param inWaitInterval the amount of time in milliseconds to wait for a message. 0 indicates
* no wait time.
*
* @return String the retrieved message
* @exception Exception thrown if an error occurred retrieving the message.
*
*/
public String getMsgFromResponseQueue(ina.mq.connector.BaseConnector connect, String inCorrelationId, int inWaitInterval) throws Exception
{
try{
// Connect to MQ
makeMQConnection( connect );
// Create response MQ Message
MQMessage myResponseMessage = new MQMessage();
// Set get options
MQGetMessageOptions myGetMessageOptions = new MQGetMessageOptions();
myGetMessageOptions.options = MQC.MQGMO_WAIT + MQC.MQGMO_FAIL_IF_QUIESCING + MQC.MQGMO_CONVERT;
// If correlation id supplied then match on it
if( inCorrelationId != null && inCorrelationId.length() > 0)
{
myGetMessageOptions.matchOptions = myGetMessageOptions.matchOptions | MQC.MQMO_MATCH_CORREL_ID;
myResponseMessage.correlationId = inCorrelationId.getBytes();
}
// Set to none
else
{
myResponseMessage.correlationId = MQC.MQCI_NONE;
}
// Set message id to none
myResponseMessage.messageId = MQC.MQCI_NONE;
// Wait time provided
if( inWaitInterval > 0)
{
// Set wait if wait option if wait time provided
myGetMessageOptions.waitInterval = inWaitInterval;
}
// Actually pull the message(s) from the MQ
mqResponseQueue.get(myResponseMessage, myGetMessageOptions);
// Read message
int dataLength = myResponseMessage.getMessageLength();
byte[] buffer = new byte[dataLength];
myResponseMessage.readFully(buffer);
String myData = new String(buffer);
finalise();
// Return message
return myData;
}catch (MQException ex)
{
System.out.println("MQ exception occurred : Completion code "
+ ex.completionCode
+ " Reason code "
+ ex.reasonCode);
log.fatal("MQ exception occurred : Completion code "
+ ex.completionCode
+ " Reason code "
+ ex.reasonCode);
//Close all connections
finalise();
// Throw exception
throw ex;
}
catch (java.io.IOException ex)
{
System.out.println("An error occurred reading from message buffer: " + ex);
log.fatal("An error occurred reading from message buffer: " +ex);
//Close all connections
finalise();
// Throw exception
throw ex;
}
} |
|
Back to top |
|
 |
fjb_saper |
Posted: Sat Sep 30, 2006 5:29 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Well some things remain quite hidden here: Are you running in a stand alone JVM or in a Web Application Server? If the latter is the case why did you not use JMS? I do hope you realize that your current code is going to incurr the overhead of the connection on every put... You did not show how you are acquiring the connection either. I suspect you are running into concurrency issues as well. Are you pooling the connections using the MQ provided pool or your own?
Like I said I would try to go for this using JMS and not base MQ.
Enjoy
 _________________ MQ & Broker admin |
|
Back to top |
|
 |
ora |
Posted: Fri Oct 06, 2006 11:51 am Post subject: |
|
|
Newbie
Joined: 20 Sep 2006 Posts: 5
|
grand master,
Service will be on Tomcat (Deployed using Axis). I am not using JMS because I need to be able to use service with a php application.
You're correct when you said "I suspect you are running into concurrency issues as well". I am definitely having a problem in this area. I am using the connection pooling of MQ. Because I am so new at this, I am not sure what you by "Are you pooling the connections using the MQ provided pool or your own?". Please explain. Also, can you tell me what will cause concurrency problems?
I have copied my connection code below.
public String makeMQConnection(ina.mq.connector.BaseConnector connect) throws Exception
{
mqQueueMgrName = connect.getMQQueueMgrName();
mqRequestQueueName = connect.getMQRequestQueueName();
mqResponseQueueName = connect.getMQResponseQueueName();
mqChannel = connect.getMQChannel();
mqHost = connect.getMQHost();
mqPort = connect.getMQPort();
// Connect to queue manager
String status = null;
try{
connectToQMgr();
// Connect to queues
connectToRequestQ();
connectToResponseQ();
// Connected successfully
status = "successful";
} catch(MQException ex){
log.info("MQ Exception occurred while connecting" + ex);
status = "failed";
} catch(Exception ex){
log.fatal("An error occurred while connecting" + ex);
status = "failed";
}
return status;
}
/**
* Sets the environment variables for MQSeries and connects to queue manager.
* The MQ host, channel and port must be set.
*
* @exception Exception thrown if an error occurred connecting to the queue manager
* or if the host, channel or port is not set.
*/
private void connectToQMgr() throws Exception
{
// Check parameters
if( mqHost == null || mqHost.length() == 0 )
{
// Throw exception
throw new Exception( "MQ Host must be specified." );
}
if( mqChannel == null || mqChannel.length() == 0 )
{
// Throw exception
throw new Exception( "MQ Channel must be specified." );
}
if( mqPort == 0 )
{
// Throw exception
throw new Exception( "MQ listening port must be specified." );
}
try
{
//Create a connection pool to the queue manager
MQSimpleConnectionManager myConnMan = new MQSimpleConnectionManager();
myConnMan.setActive(MQSimpleConnectionManager.MODE_AUTO);
myConnMan.setTimeout(2000);
MQEnvironment.setDefaultConnectionManager(myConnMan);
//Register a token
token = MQEnvironment.addConnectionPoolToken();
// Set environment (connection) variables
mqEnv.put(MQC.HOST_NAME_PROPERTY, mqHost);
mqEnv.put(MQC.CHANNEL_PROPERTY, mqChannel);
mqEnv.put(MQC.PORT_PROPERTY, new Integer(mqPort));
mqEnv.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES);
// Connect to queue manager
mqQueueMgr = new MQQueueManager(mqQueueMgrName, mqEnv);
}
catch(MQException ex)
{
try
{
if(mqQueueMgr != null)
{
mqQueueMgr.close();
}
}
catch(Exception ignored)
{
}
try
{
if(mqQueueMgr != null)
{
mqQueueMgr.disconnect();
}
}
catch(Exception ignored)
{
}
throw ex;
}
}
/**
* Connects to request queue using the basic MQ options. If an exception occurs
* connecting to the request queue it is closed because any object returned is
* not valid.
*
* @exception Exception thrown if an error occurred connecting to the request queue.
*/
private void connectToRequestQ() throws Exception
{
try
{
int openOptions = MQC.MQOO_OUTPUT+MQC.MQOO_INQUIRE + MQC.MQOO_FAIL_IF_QUIESCING + MQC.MQOO_INPUT_SHARED;
mqRequestQueue = mqQueueMgr.accessQueue(mqRequestQueueName, openOptions, null, null, null);
}
catch(MQException ex)
{
try
{
if(mqRequestQueue != null)
{
mqRequestQueue.close();
}
}
catch(Exception ignored)
{
}
// Throw exception
throw ex;
}
}
/**
* Connects to response queue using the basic MQ options. If an exception occurs
* connecting to the response queue it is closed because any object returned is
* not valid.
*
* @exception Exception thrown if an error occurred connecting to the response queue.
*/
private void connectToResponseQ() throws Exception
{
try
{
int openOptions = MQC.MQOO_OUTPUT+MQC.MQOO_INQUIRE + MQC.MQOO_FAIL_IF_QUIESCING + MQC.MQOO_INPUT_SHARED;
mqResponseQueue = mqQueueMgr.accessQueue(mqResponseQueueName, openOptions, null, null, null);
}
catch(MQException ex)
{
try
{
if(mqResponseQueue != null)
{
mqResponseQueue.close();
}
}
catch(Exception ignored)
{
}
// Throw exception
throw ex;
}
} |
|
Back to top |
|
 |
fjb_saper |
Posted: Fri Oct 06, 2006 2:43 pm Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Well some of the concurrency issues will come from the fact that the MQEnvironment class is a singleton...
I would define a file JNDI and run as JMS. Regardless of the fact that you're targeting a php server your code still seems to be in Java so why not make it JMS and have an easy way of handling concurrency issues....
In JMS for each thread you need to:
- grab a connection from the pool (done through jndi) by getting the qcf and instanciating a connection off the qcf.
- instanciate a session from the connection
- instanciate the queues from jndi
- instanciate producers and consumers from the session
- don't forget to start the connection before receiving
- don't forget to close the objects in the reverse order of creation
See no concurrency problem... each thread has its own stuff
Enjoy  _________________ MQ & Broker admin |
|
Back to top |
|
 |
ora |
Posted: Thu Oct 12, 2006 10:56 am Post subject: |
|
|
Newbie
Joined: 20 Sep 2006 Posts: 5
|
Thank you grand master.
I was able to resolve the issue with making concurrent calls. The issue was due to declaring my connection variables as global variables. I have also changed my connection pool so that it uses mq default connection pool.
Even though I made this change we are still having problems with getting all of the response back. We might put 20 concurrent msgs out on the request queue with an wait interval of 5 sec but get back 4 to 5 msgs. When we increment our wait interval (example 25 secs) and put 20 concurrent msgs on request queue we get replies for all 20 msgs. It takes a total of 22 secs to process all msgs. It seems that the time a reply is available is based on the depth of the request queue. The wait interval is not consistent for each request. example one message comes back in 3 secs and another 6 secs. How can we avoid this inconsistance. |
|
Back to top |
|
 |
fjb_saper |
Posted: Thu Oct 12, 2006 3:06 pm Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Well there is no inconsistency. If you put 20 messages on the queue and there is only one process reading them it will take 20*time for 1 message to process them all...
This is an application design and scalability problem. Unless you tell us more about the server application there is nothing we can do.
Enjoy  _________________ MQ & Broker admin |
|
Back to top |
|
 |
ora |
Posted: Fri Oct 13, 2006 6:44 am Post subject: |
|
|
Newbie
Joined: 20 Sep 2006 Posts: 5
|
grand master,
MQ is a clients remote MQ (CICS). Client uses a trigger on the request queue. When I put a message on request queue, a trigger is set so that a reply is put on the response queue.
I was told the following by client: Trigger type is first. The server application (Cobol) that processes request grabs all messages on queue when it is first triggered. The program loops through messages and commits each proccess independently and puts messages back on queue. The application get request under syncpoint and puts reply under syncpoint. (Only one program is running)
this is the result of 500 message test.
Start Time : MSG GET 001 2006-10-12-15.33.01.845820
End Time : MSG PUT 500 2006-10-12-15.34.10.324633
Total Time : 69 seconds.
Please keep in mind the 500 messages were put on request queue concurrently |
|
Back to top |
|
 |
fjb_saper |
Posted: Fri Oct 13, 2006 2:45 pm Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Like I said: Design and scaling problem.
You might want to Tclass the MF process and set trigger to every. You cannot do that if you did not TClass the CICS transaction as you would be killing the CICS process hogging all the resources...
Talk to and with your partners. IT IS A MUST.
Enjoy  _________________ MQ & Broker admin |
|
Back to top |
|
 |
ora |
Posted: Mon Oct 23, 2006 5:19 am Post subject: |
|
|
Newbie
Joined: 20 Sep 2006 Posts: 5
|
Grand Master,
Thanks for your input. I am looking into using jms. One more question before I do. What will cause my script to take about 40 secs to connect to the queue manager (this is per message). This occurs at the point of connection.
FYI: I am using MQSimpleConnectionPool for connection pooling.
Ora
Revised function:
private void connectToQMgr() throws Exception
{
// Check parameters
if( mqHost == null || mqHost.length() == 0 )
{
// Throw exception
throw new Exception( "MQ Host must be specified." );
}
if( mqChannel == null || mqChannel.length() == 0 )
{
// Throw exception
throw new Exception( "MQ Channel must be specified." );
}
if( mqPort == 0 )
{
// Throw exception
throw new Exception( "MQ listening port must be specified." );
}
try
{
//Create a connection pool to the queue manager
MQSimpleConnectionManager myConnMan = new MQSimpleConnectionManager();
myConnMan.setActive(MQSimpleConnectionManager.MODE_AUTO);
MQEnvironment.setDefaultConnectionManager(myConnMan);
// Set environment (connection) variables
mqEnv.put(MQC.HOST_NAME_PROPERTY, mqHost);
mqEnv.put(MQC.CHANNEL_PROPERTY, mqChannel);
mqEnv.put(MQC.PORT_PROPERTY, new Integer(mqPort));
mqEnv.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES);
// Connect to queue manager
mqQueueMgr = new MQQueueManager(mqQueueMgrName, mqEnv, myConnMan);
}
catch(MQException ex)
{
try
{
if(mqQueueMgr != null)
{
mqQueueMgr.close();
}
}
catch(Exception ignored)
{
}
try
{
if(mqQueueMgr != null)
{
mqQueueMgr.disconnect();
}
}
catch(Exception ignored)
{
}
throw ex;
}
} |
|
Back to top |
|
 |
fjb_saper |
Posted: Mon Oct 23, 2006 2:52 pm Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
ora wrote: |
Grand Master,
Thanks for your input. I am looking into using jms. One more question before I do. What will cause my script to take about 40 secs to connect to the queue manager (this is per message). This occurs at the point of connection.
|
Your network maybe ? A starved and inadequate pool? _________________ MQ & Broker admin |
|
Back to top |
|
 |
|