ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Java / JMS » Issue w/ concurrent get calls using java class and web ser

Post new topic  Reply to topic
 Issue w/ concurrent get calls using java class and web ser « View previous topic :: View next topic » 
Author Message
ora
PostPosted: Thu Sep 21, 2006 1:59 pm    Post subject: Issue w/ concurrent get calls using java class and web ser Reply with quote

Newbie

Joined: 20 Sep 2006
Posts: 5

Hello,

I have a java class that i used to create a web service. I have one function (putMsgOnRequestQueue) that puts a message on to a request queue and another function that gets message off of a response queue. When I run my script with java (main), I know that I can put and get a successful response. Even when I run a test using a java Client that calls web service, I can put and get a successful response. The issue is when I do multiple calls to web service at a time. Example: 5 concurrent calls. I might get back the first couple of responses then I get null pointer exception (Error 2033 - NO MSG ON QUEUE). When I close queues and disconnect from queue manager, message is on response queue. I have looked into adjusting my wait interval and this problem still occurs. I can see all of my requests on the request queue, so putting message on request queue doesn't seem to be a problem. It is when I attempt to get a message from the response queue. I am not certain of this but, issue seems to be related to using a priority of some sort when responding to request.


FYI: MQ is a clients remote MQ (CICS). Client uses a trigger on the request queue. When I put a message on request queue, a trigger is set so that a reply is put on the response queue.

Java put and get functions below:

/**

* This method puts a message on the Request Queue and returns the correlation

* id.

*

* @param inMsg the message to send.

* @return String the correlation id.

* @exception Exception thrown if an error occurred sending/receiving the message.

*

*/

public String putMsgOnRequestQueue(ina.mq.connector.BaseConnector connect, String inMsg) throws Exception

{

try

{



// Connect to MQ

makeMQConnection( connect );



// Create MQ Message

MQMessage myRequestMsg = new MQMessage();



// create message options

MQPutMessageOptions pmo = new MQPutMessageOptions();



// accept defaults

pmo.options = MQC.MQPMO_NONE;



// Clear the message

myRequestMsg.clearMessage();



// Get next correlation id

getNextCorrelationId();



// Set correlation id

myRequestMsg.correlationId = mqCorrelationId.getBytes();



// Set message id

myRequestMsg.messageId = mqCorrelationId.getBytes();



// Set message format

myRequestMsg.format = MQC.MQFMT_STRING;



// Write the message

myRequestMsg.writeString(inMsg);



// Put message on the queue

mqRequestQueue.put(myRequestMsg, pmo);



finalise();

}

catch (MQException ex)

{

System.out.println("MQ exception occurred : Completion code "

+ ex.completionCode

+ " Reason code "

+ ex.reasonCode);



log.fatal("MQ exception occurred : Completion code "

+ ex.completionCode

+ " Reason code "

+ ex.reasonCode);

//Close all connections

finalise();



// Throw exception

throw ex;

}

catch (java.io.IOException ex)

{

System.out.println("An error occurred reading from message buffer: " + ex);

log.fatal("An error occurred reading from message buffer: " +ex);



//Close all connections

finalise();



// Throw exception

throw ex;

}



// Return correlation id

return mqCorrelationId;

}



/**

* This method retrieves a message from the Response Queue based on the correlation

* id. If the correlation id is null then the next message on the queue is

* retrieved.

*

* @param inCorrelationId the correlation id of the message to retrieve or null

* to retrieve the next message.

* @param inWaitInterval the amount of time in milliseconds to wait for a message. 0 indicates

* no wait time.

*

* @return String the retrieved message

* @exception Exception thrown if an error occurred retrieving the message.

*

*/

public String getMsgFromResponseQueue(ina.mq.connector.BaseConnector connect, String inCorrelationId, int inWaitInterval) throws Exception

{



try{

// Connect to MQ

makeMQConnection( connect );





// Create response MQ Message

MQMessage myResponseMessage = new MQMessage();



// Set get options

MQGetMessageOptions myGetMessageOptions = new MQGetMessageOptions();

myGetMessageOptions.options = MQC.MQGMO_WAIT + MQC.MQGMO_FAIL_IF_QUIESCING + MQC.MQGMO_CONVERT;



// If correlation id supplied then match on it

if( inCorrelationId != null && inCorrelationId.length() > 0)

{

myGetMessageOptions.matchOptions = myGetMessageOptions.matchOptions | MQC.MQMO_MATCH_CORREL_ID;

myResponseMessage.correlationId = inCorrelationId.getBytes();

}

// Set to none

else

{

myResponseMessage.correlationId = MQC.MQCI_NONE;

}



// Set message id to none

myResponseMessage.messageId = MQC.MQCI_NONE;





// Wait time provided

if( inWaitInterval > 0)

{

// Set wait if wait option if wait time provided

myGetMessageOptions.waitInterval = inWaitInterval;

}



// Actually pull the message(s) from the MQ

mqResponseQueue.get(myResponseMessage, myGetMessageOptions);



// Read message

int dataLength = myResponseMessage.getMessageLength();

byte[] buffer = new byte[dataLength];

myResponseMessage.readFully(buffer);

String myData = new String(buffer);



finalise();

// Return message

return myData;



}catch (MQException ex)

{

System.out.println("MQ exception occurred : Completion code "

+ ex.completionCode

+ " Reason code "

+ ex.reasonCode);



log.fatal("MQ exception occurred : Completion code "

+ ex.completionCode

+ " Reason code "

+ ex.reasonCode);

//Close all connections

finalise();



// Throw exception

throw ex;

}

catch (java.io.IOException ex)

{

System.out.println("An error occurred reading from message buffer: " + ex);

log.fatal("An error occurred reading from message buffer: " +ex);



//Close all connections

finalise();



// Throw exception

throw ex;

}

}
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sat Sep 30, 2006 5:29 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

Well some things remain quite hidden here: Are you running in a stand alone JVM or in a Web Application Server? If the latter is the case why did you not use JMS? I do hope you realize that your current code is going to incurr the overhead of the connection on every put... You did not show how you are acquiring the connection either. I suspect you are running into concurrency issues as well. Are you pooling the connections using the MQ provided pool or your own?

Like I said I would try to go for this using JMS and not base MQ.

Enjoy


_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
ora
PostPosted: Fri Oct 06, 2006 11:51 am    Post subject: Reply with quote

Newbie

Joined: 20 Sep 2006
Posts: 5

grand master,

Service will be on Tomcat (Deployed using Axis). I am not using JMS because I need to be able to use service with a php application.

You're correct when you said "I suspect you are running into concurrency issues as well". I am definitely having a problem in this area. I am using the connection pooling of MQ. Because I am so new at this, I am not sure what you by "Are you pooling the connections using the MQ provided pool or your own?". Please explain. Also, can you tell me what will cause concurrency problems?


I have copied my connection code below.


public String makeMQConnection(ina.mq.connector.BaseConnector connect) throws Exception

{



mqQueueMgrName = connect.getMQQueueMgrName();

mqRequestQueueName = connect.getMQRequestQueueName();

mqResponseQueueName = connect.getMQResponseQueueName();

mqChannel = connect.getMQChannel();

mqHost = connect.getMQHost();

mqPort = connect.getMQPort();



// Connect to queue manager

String status = null;



try{

connectToQMgr();



// Connect to queues

connectToRequestQ();

connectToResponseQ();



// Connected successfully

status = "successful";

} catch(MQException ex){



log.info("MQ Exception occurred while connecting" + ex);

status = "failed";



} catch(Exception ex){



log.fatal("An error occurred while connecting" + ex);

status = "failed";

}



return status;



}

/**

* Sets the environment variables for MQSeries and connects to queue manager.

* The MQ host, channel and port must be set.

*

* @exception Exception thrown if an error occurred connecting to the queue manager

* or if the host, channel or port is not set.

*/

private void connectToQMgr() throws Exception

{

// Check parameters

if( mqHost == null || mqHost.length() == 0 )

{

// Throw exception

throw new Exception( "MQ Host must be specified." );

}

if( mqChannel == null || mqChannel.length() == 0 )

{

// Throw exception

throw new Exception( "MQ Channel must be specified." );

}

if( mqPort == 0 )

{

// Throw exception

throw new Exception( "MQ listening port must be specified." );

}



try

{

//Create a connection pool to the queue manager

MQSimpleConnectionManager myConnMan = new MQSimpleConnectionManager();

myConnMan.setActive(MQSimpleConnectionManager.MODE_AUTO);

myConnMan.setTimeout(2000);

MQEnvironment.setDefaultConnectionManager(myConnMan);



//Register a token

token = MQEnvironment.addConnectionPoolToken();



// Set environment (connection) variables

mqEnv.put(MQC.HOST_NAME_PROPERTY, mqHost);

mqEnv.put(MQC.CHANNEL_PROPERTY, mqChannel);

mqEnv.put(MQC.PORT_PROPERTY, new Integer(mqPort));

mqEnv.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES);



// Connect to queue manager

mqQueueMgr = new MQQueueManager(mqQueueMgrName, mqEnv);

}

catch(MQException ex)

{

try

{

if(mqQueueMgr != null)

{

mqQueueMgr.close();

}

}

catch(Exception ignored)

{

}

try

{

if(mqQueueMgr != null)

{

mqQueueMgr.disconnect();

}

}

catch(Exception ignored)

{

}

throw ex;

}

}



/**

* Connects to request queue using the basic MQ options. If an exception occurs

* connecting to the request queue it is closed because any object returned is

* not valid.

*

* @exception Exception thrown if an error occurred connecting to the request queue.

*/

private void connectToRequestQ() throws Exception

{

try

{

int openOptions = MQC.MQOO_OUTPUT+MQC.MQOO_INQUIRE + MQC.MQOO_FAIL_IF_QUIESCING + MQC.MQOO_INPUT_SHARED;

mqRequestQueue = mqQueueMgr.accessQueue(mqRequestQueueName, openOptions, null, null, null);

}

catch(MQException ex)

{

try

{

if(mqRequestQueue != null)

{

mqRequestQueue.close();

}

}

catch(Exception ignored)

{

}



// Throw exception

throw ex;

}

}



/**

* Connects to response queue using the basic MQ options. If an exception occurs

* connecting to the response queue it is closed because any object returned is

* not valid.

*

* @exception Exception thrown if an error occurred connecting to the response queue.

*/

private void connectToResponseQ() throws Exception

{

try

{

int openOptions = MQC.MQOO_OUTPUT+MQC.MQOO_INQUIRE + MQC.MQOO_FAIL_IF_QUIESCING + MQC.MQOO_INPUT_SHARED;

mqResponseQueue = mqQueueMgr.accessQueue(mqResponseQueueName, openOptions, null, null, null);

}

catch(MQException ex)

{

try

{

if(mqResponseQueue != null)

{

mqResponseQueue.close();

}

}

catch(Exception ignored)

{

}



// Throw exception

throw ex;

}

}
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Oct 06, 2006 2:43 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

Well some of the concurrency issues will come from the fact that the MQEnvironment class is a singleton...

I would define a file JNDI and run as JMS. Regardless of the fact that you're targeting a php server your code still seems to be in Java so why not make it JMS and have an easy way of handling concurrency issues....

In JMS for each thread you need to:
  1. grab a connection from the pool (done through jndi) by getting the qcf and instanciating a connection off the qcf.
  2. instanciate a session from the connection
  3. instanciate the queues from jndi
  4. instanciate producers and consumers from the session
  5. don't forget to start the connection before receiving
  6. don't forget to close the objects in the reverse order of creation


See no concurrency problem... each thread has its own stuff

Enjoy
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
ora
PostPosted: Thu Oct 12, 2006 10:56 am    Post subject: Reply with quote

Newbie

Joined: 20 Sep 2006
Posts: 5

Thank you grand master.

I was able to resolve the issue with making concurrent calls. The issue was due to declaring my connection variables as global variables. I have also changed my connection pool so that it uses mq default connection pool.

Even though I made this change we are still having problems with getting all of the response back. We might put 20 concurrent msgs out on the request queue with an wait interval of 5 sec but get back 4 to 5 msgs. When we increment our wait interval (example 25 secs) and put 20 concurrent msgs on request queue we get replies for all 20 msgs. It takes a total of 22 secs to process all msgs. It seems that the time a reply is available is based on the depth of the request queue. The wait interval is not consistent for each request. example one message comes back in 3 secs and another 6 secs. How can we avoid this inconsistance.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Thu Oct 12, 2006 3:06 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

Well there is no inconsistency. If you put 20 messages on the queue and there is only one process reading them it will take 20*time for 1 message to process them all...

This is an application design and scalability problem. Unless you tell us more about the server application there is nothing we can do.

Enjoy
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
ora
PostPosted: Fri Oct 13, 2006 6:44 am    Post subject: Reply with quote

Newbie

Joined: 20 Sep 2006
Posts: 5

grand master,

MQ is a clients remote MQ (CICS). Client uses a trigger on the request queue. When I put a message on request queue, a trigger is set so that a reply is put on the response queue.

I was told the following by client: Trigger type is first. The server application (Cobol) that processes request grabs all messages on queue when it is first triggered. The program loops through messages and commits each proccess independently and puts messages back on queue. The application get request under syncpoint and puts reply under syncpoint. (Only one program is running)

this is the result of 500 message test.

Start Time : MSG GET 001 2006-10-12-15.33.01.845820
End Time : MSG PUT 500 2006-10-12-15.34.10.324633

Total Time : 69 seconds.

Please keep in mind the 500 messages were put on request queue concurrently
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Oct 13, 2006 2:45 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

Like I said: Design and scaling problem.

You might want to Tclass the MF process and set trigger to every. You cannot do that if you did not TClass the CICS transaction as you would be killing the CICS process hogging all the resources...

Talk to and with your partners. IT IS A MUST.

Enjoy
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
ora
PostPosted: Mon Oct 23, 2006 5:19 am    Post subject: Reply with quote

Newbie

Joined: 20 Sep 2006
Posts: 5

Grand Master,

Thanks for your input. I am looking into using jms. One more question before I do. What will cause my script to take about 40 secs to connect to the queue manager (this is per message). This occurs at the point of connection.

FYI: I am using MQSimpleConnectionPool for connection pooling.

Ora

Revised function:

private void connectToQMgr() throws Exception
{

// Check parameters

if( mqHost == null || mqHost.length() == 0 )
{

// Throw exception
throw new Exception( "MQ Host must be specified." );

}

if( mqChannel == null || mqChannel.length() == 0 )
{

// Throw exception
throw new Exception( "MQ Channel must be specified." );

}

if( mqPort == 0 )
{

// Throw exception
throw new Exception( "MQ listening port must be specified." );

}



try
{

//Create a connection pool to the queue manager

MQSimpleConnectionManager myConnMan = new MQSimpleConnectionManager();

myConnMan.setActive(MQSimpleConnectionManager.MODE_AUTO);
MQEnvironment.setDefaultConnectionManager(myConnMan);

// Set environment (connection) variables

mqEnv.put(MQC.HOST_NAME_PROPERTY, mqHost);
mqEnv.put(MQC.CHANNEL_PROPERTY, mqChannel);
mqEnv.put(MQC.PORT_PROPERTY, new Integer(mqPort));
mqEnv.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES);



// Connect to queue manager

mqQueueMgr = new MQQueueManager(mqQueueMgrName, mqEnv, myConnMan);

}

catch(MQException ex)
{

try
{

if(mqQueueMgr != null)
{
mqQueueMgr.close();
}

}

catch(Exception ignored)
{

}

try
{

if(mqQueueMgr != null)
{
mqQueueMgr.disconnect();
}

}
catch(Exception ignored)
{

}

throw ex;

}

}
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Oct 23, 2006 2:52 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

ora wrote:
Grand Master,

Thanks for your input. I am looking into using jms. One more question before I do. What will cause my script to take about 40 secs to connect to the queue manager (this is per message). This occurs at the point of connection.

Your network maybe ? A starved and inadequate pool?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » IBM MQ Java / JMS » Issue w/ concurrent get calls using java class and web ser
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.