ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » Clustering » Conversion in a cluster

Post new topic  Reply to topic Goto page 1, 2  Next
 Conversion in a cluster « View previous topic :: View next topic » 
Author Message
mattfarney
PostPosted: Mon Apr 04, 2011 9:51 am    Post subject: Conversion in a cluster Reply with quote

Disciple

Joined: 17 Jan 2006
Posts: 167
Location: Ohio

I have an odd scenario. I have a mix of windows, unix, and z/OS machines. The windows and unix machines are clustered comfortably. Now, I am trying to include the z/OS machines in the cluster, but all of their existing channels have convert(YES) due to the way the applications were coded 10-15 years ago.

I've seen discussions that using conversion is possible in a cluster, but not recommended. I'm working under the assumption that the applications will not be modified to do read-time conversion - so I'm left with using convert in the cluster or not clustering.

Assuming that all of the boxes can interact, I believe I will need to set every cluster receiver with convert(yes) along with the explicit cluster senders from the z/OS machines to the full repositories.

Am I missing anything?

How much real overhead is created by using convert(yes)?

-mf
Back to top
View user's profile Send private message
bruce2359
PostPosted: Mon Apr 04, 2011 10:05 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

Quote:
I've seen discussions that using conversion is possible in a cluster, but not recommended.

MQGMO_CONVERT is the IBM-recommended way of dealing with messages that might be in need of conversion. MQGMO_CONVERT means that the consumer of the message drives the conversion process, if necessary.

CONVERT(YES) is an alternative - a valid alternative. Go ahead and use it. No worries.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Mon Apr 04, 2011 12:53 pm    Post subject: Re: Conversion in a cluster Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

mattfarney wrote:
How much real overhead is created by using convert(yes)?

There's no additional overhead required to perform one conversion in the channel, as opposed to performing it somewhere else. Any additional overhead would be due to the channel performing unnecessary conversions. (For example, if the receiving application also converts on get, then the channel conversion would be redundant.) The performance impact may or may not be significant.

Note that the receiving qmgr determines the destination ccsid; the receiving app has no control over that, (except by performing a second conversion).

Also note that neither the sending or receiving app can detect any channel conversion errors. If the channel fails to convert a message, the receiving qmgr will never see it, and it will likely end up on the sending qmgr dead letter queue.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Mon Apr 04, 2011 1:08 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

I would not be so gun-ho about this. All messages in the cluster will now be converted as they pass between QMs. In the cluster the sending and receiving QMs will talk directly to each other, and you won't be getting multi-hops thru the QMs, so you probably won't be seeing the same messages being converted multiple times.

But there may be apps on the non mainframe servers that were built assuming no conversion was being done. They may require the messages unconverted (I'm not saying they are correctly built!). Now all of the sudden the messages are showing up in the queue converted. Uh-oh.

What if there are custom conversion exits on some of the QMs? And some unique MQMD_FORMATs flying around? You might find problems with channels trying to convert these.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
bruce2359
PostPosted: Mon Apr 04, 2011 1:09 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.



Stated differently: If the message needs to be converted, it will be converted either on the sending end with CONVERT(YES), or by the receiving end by MQGMO_CONVERT.

Basic (EBCDIC to ASCII) conversion on either end is a simple table lookup that converts byte-by-byte. There is no heavy cpu utilization.

If the receiving end has a conversion utility exit in place, the exit might be imposing some additional cpu time. You would see this if the sender channel definition had something like MSGEXIT(myexit); or the msg MQMD contains an application name in the MQFMT_ field.

If the apps are already written, there is no real reason to change their behavior.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Mon Apr 04, 2011 1:14 pm    Post subject: Re: Conversion in a cluster Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

mattfarney wrote:
Assuming that all of the boxes can interact, I believe I will need to set every cluster receiver with convert(yes) along with the explicit cluster senders from the z/OS machines to the full repositories.


I would think you would only need and want it on the mainframe CLUSRCVR channels.

If the mainframe apps send to the Unix / Windows QMs, all the apps there are written to convert on the MQGET, so no need for the channels into and between those QMs to be converting.

The channels into and between the mainframe QMs would need the conversion on the channels, so only set it for them.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
mqjeff
PostPosted: Mon Apr 04, 2011 2:44 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

You really need to think twice about enabling CONVERT on any cluster channel.

Best bet if you really need to use CONVERT for specific mainframe applications is to create QREMOTES on one or two distributed qmgrs that are shared in the cluster. Then these can point to an XMITQ that is read by a specific non-cluster SDR channel that will have CONVERT enabled on it's RCVR.
Back to top
View user's profile Send private message
zpat
PostPosted: Mon Apr 04, 2011 11:58 pm    Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5849
Location: UK

I thought CONVERT was set on the SENDER channel, not the receiver channel?
Back to top
View user's profile Send private message
skoobee
PostPosted: Tue Apr 05, 2011 12:20 am    Post subject: Reply with quote

Acolyte

Joined: 26 Nov 2010
Posts: 52

Attributes are always copied from the CLUSRCVR to the CLUSSDR, even when the CLUSSDR is manually defined.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Tue Apr 05, 2011 12:27 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

zpat wrote:
I thought CONVERT was set on the SENDER channel, not the receiver channel?

From the Info Center:
Quote:
This parameter is valid only for channels with a channel type (CHLTYPE) of SDR, SVR, CLUSSDR, or CLUSRCVR.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Apr 05, 2011 12:58 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

skoobee wrote:
Attributes are always copied from the CLUSRCVR to the CLUSSDR, even when the CLUSSDR is manually defined.

More accurately, the explicitly-defined CLUSSDR is discarded, and replaced with a CLUSSDRB-type channel - with identical attributes.

Automatically defined CLUSSDRA-type channels take their attributes from the CLUSRCVR definition.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Apr 05, 2011 8:08 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

From the OP:

Quote:
...I have a mix of windows, unix, and z/OS machines. The windows and unix machines are clustered comfortably. Now, I am trying to include the z/OS machines in the cluster, but all of their existing channels have convert(YES) due to the way the applications were coded 10-15 years ago.

Yes, and I surmise that the apps are functioning properly.

Quote:
I've seen discussions that using conversion is possible in a cluster, but not recommended. I'm working under the assumption that the applications will not be modified to do read-time conversion - so I'm left with using convert in the cluster or not clustering.

You are already using CONVERT(YES) according to your first paragraph.

Quote:
Assuming that all of the boxes can interact, I believe I will need to set every cluster receiver with convert(yes) along with the explicit cluster senders from the z/OS machines to the full repositories.

Again, you indicated in the first paragraph that CONVERT(YES) is already in place.

Conversion taking place on the sending end of the channel will drive conversion from the sending platform CCSID to the receiving platform CCSID - as it is currently doing from Win to UNIX. If z/OS qmgr joins the cluster, the same process will take place.

So, if a msg leaves a UNIX qmgr, the sending end MCA will drive conversion to the z/OS recieving end CCSID. The same holds true if the msg is sent from a z/OS qmgr, and arrives at a Win or UNIX qmgr.

It is not unusual to have a mix of Win, UNIX and z/OS qmgrs in the same cluster. If conversion is simple (ASCII to EBCDIC, for example), all should work fine.

If the app data is platform-sensitive, AND a user-written conversion exit is currently used to drive the conversion, then integrating a z/OS qmgr into the cluster might be a problem.

Further analysis on your part is in order.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Apr 05, 2011 10:52 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

There is also one big problem nobody is addressing right now.
Conversion of future messages and receiving invalid messages.

Imagine a message in CCSID 500 that the receiving application wants to parse in CCSID 1208 but the qmgr is CCSID 819 and there are characters on the message that do not translate to CCSID 819...

If this is an XML message you would get a malformed XML message on the receiving qmgr...

Remember that setting convert yes to the distributed clusterrcvrs will also affect messages coming from distributed qmgrs (same ccsid problems and malformed XML problems)

So conversion to the MF, if needed yes. Conversion from the MF to distributed I would suggest to examine specifically each case and go with Jeff's suggestion of an additional non clustered channel that allows for specific conversion maybe to a qmgr with CCSID 1208?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
gbaddeley
PostPosted: Tue Apr 05, 2011 3:40 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

Best practise is not to use CONVERT(YES) on cluster channels, for reasons already posted.

If there is a requirement to use CONVERT(YES) because some legacy or badly written apps require cross-platform messaging but did not specify MQGMO_CONVERT, it is recommended to set up special sender/receiver channels and transmission queues for use by these apps.

I have seen this approach used on a large government agency.

Hammer the developers to fix their apps.

HTH,
_________________
Glenn
Back to top
View user's profile Send private message
rekarm01
PostPosted: Wed Apr 06, 2011 1:06 am    Post subject: Re: Conversion in a cluster Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

fjb_saper wrote:
Imagine a message in CCSID 500 that the receiving application wants to parse in CCSID 1208 but the qmgr is CCSID 819 and there are characters on the message that do not translate to CCSID 819...

Good point, but bad example. CCSID 500 and CCSID 819 use the same character set. There aren't any characters that will fail to translate from one to the other. How about converting to/from CCSID 437 instead? That's almost guaranteed to cause problems ...

fjb_saper wrote:
If this is an XML message you would get a malformed XML message on the receiving qmgr...

Although with channel conversion, the XML message is more likely to end up on the sending qmgr's dead letter queue, where it won't cause any problems at all for the receiver.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » Clustering » Conversion in a cluster
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.