| Author |
Message
|
| haeberm |
 Posted: Fri Jan 09, 2009 3:10 am Post subject: can I re-parse a complete message within a flow? |
 |
|
Newbie
Joined: 23 Dec 2008
Posts: 9
|
Hello, I have the following problem: In a first flow, I read a message from an MQInput node and store the complete message in a DB blob column. Then in a second flow, I have to reconstruct the message from the database. I tried this using something like
| Code: |
| CREATE LASTCHILD OF OutputRoot DOMAIN('BLOB') PARSE(newMsg); |
where newMsg is the BLOB read from the database. This works fine when I propagate this to an MQOutput node without further processing.
However, before propagation I need to alter the MQRFH2 header of the reconstructed message, so basically I want the message to be reparsed like it would be done by the MQInput node so that I can access the headers of the message.
Is this possible somehow? |
|
| Back to top |
|
 |
| kimbert |
| Posted: Fri Jan 09, 2009 5:43 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| In a first flow, I read a message from an MQInput node and store the complete message in a DB blob column |
How are you doing this ( Warehouse node perhaps?)
Have you followed good practice, and stored the code page and encoding of the bitstream along with the bytes?
Normally, I would suggest that you use CREATE...PARSE or an RCD node. But those techniques only work for the body of the message, not for the entire message (headers+body).
There is no straightforward way to re-parse an entire message including all of its headers within a message flow. That is the responsibility of input nodes only. So the only solution is to write the message out to a queue and parse it using another MQInput node ( in other words, do the task in a separate message flow ). |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Jan 09, 2009 8:15 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
You are on the right track.
I have done this several times.
In these systems, I store the following items in separate columns.
- Properties
- MQMD
- InputRoot
- MQRFH2 ( Optional in MQMD.Format is set to indicate there is one present)
By saving all this, you can reteive everything you need to reconstruct the message.
When you retrieve the message you then build a new message and then propagate it.
Beware of memory limits if you propagate too many messages in one UOW.
You can create the output message tree by a succession of :-
SET OutputRoot = NULL;
SET OutputRoot.Properties = ....
Create LASTCHILD of OutputRoot DOMAIN 'MQMD' PARSE(location where you read the table.MQMD)
--add MQRFH2 header here is present
Create LASTCHILD of OutputRoot DOMAIN 'BLOB' PARSE (...BLOB)
PROPAGATE
It does depend upon how you save the data in the first place.
I generally save the input message folders as BLOBs.
for example
set InputMQMD = asbitstream(InputRoot.MQMD);
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| haeberm |
| Posted: Fri Jan 09, 2009 2:15 pm Post subject: |
|
|
Newbie
Joined: 23 Dec 2008
Posts: 9
|
@kimbert: Thanks for your clarification, I'll go the way that you suggested and use a intermediate queue to put the message on.
One question I have regarding the "good practice" you mention: Wouldn't it be sufficient to specify the same fixed CCSID and encoding for both the ASBITSTREAM and CREATE...PARSE functions in order to make sure that it will be working even with brokers using different codepages?
@smdavies99: Thanks for your tips. The problem why this does not work for me is that I don't know whether or not there are more MQ headers besides the MQRFH2 in the message and whether those additional headers are before or after the MQRFH2. So like kimbert said I need both headers+body being parsed.
Alternatively I could probably change the database layout in a way that each header gets stored separately together with its format (domain). |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Jan 09, 2009 2:41 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
What additional headers apart from MQRFH2 are you possibly going to receive?
There are a few possibilities but in today, the likelyhood of getting them is probably pretty rare.
I'd sugget that you look in the WMQ programming docs for info on how to handle those headers.
It is all down to how they link together.
Then you can code your ESQL just as if you were coding in 'C'.
In my time of programming WMQ (10+ years) I have only encountered one additional header (other than MQRFH2) and that was when I working with CICS messages.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Jan 09, 2009 3:15 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
smdavies' advice is good, and a colleague at IBM was suggesting the same approach. However, only haeberm knows whether or not he is likely to get 'other headers'.
mqjeff emailed me privately to point out that there is no need to use a separate flow. You can wire the MQOutput node direct to an MQGet node. That way, everything will happen in the same MQ transaction, and performance will be better.
| Quote: |
| Wouldn't it be sufficient to specify the same fixed CCSID and encoding for both the ASBITSTREAM and CREATE...PARSE functions in order to make sure that it will be working even with brokers using different codepages? |
Aha! So you're saving the bitstream using ASBITSTREAM on the input message. So the answer to your question is 'yes'. But if your chosen CCSID is not the same as the input message, your flow will burn some CPU because it will have to parse, then re-serialize the entire mesage. If you care about that, then it is another point in favour of smdavies' approach. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Jan 10, 2009 3:27 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| smdavies99 wrote: |
In my time of programming WMQ (10+ years) I have only encountered one additional header (other than MQRFH2) and that was when I working with CICS messages. |
Looks like you haven't much worked with SAP and MQLink-for R3... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Sat Jan 10, 2009 3:45 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| Quote: |
fjp_sapper wrote:
| Quote: |
smdavies99 wrote:
In my time of programming WMQ (10+ years) I have only encountered one additional header (other than MQRFH2) and that was when I working with CICS messages.
|
Looks like you haven't much worked with SAP and MQLink-for R3...
|
I have used SAP (used via WMQ SAP Adapter) but not on Mainframes but yes, you are correct. 
I was trying to say that he does not need to cater for headers that he will never encounter with his business application area.
As can be seen from the tpoics in this forum, the most common header used these days is MQRFH2 but you are correct there are others but can be possibly be handled in the same way as MQRFH2.
/S
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| haeberm |
| Posted: Tue Jan 13, 2009 3:05 am Post subject: |
|
|
Newbie
Joined: 23 Dec 2008
Posts: 9
|
Thanks a lot for all the valuable information.
The flow I am going to develop (the one storing the messages) will end up being a subflow which can be embedded by clients in their own flows. This is why I don't know which other headers (CICS, IMS,...) might be present. Only MQRFH2 is mandatory.
I decided to redesign my database layout and to add an additional table to store each part of the message (headers + body) individually together with the parser (obtained by "... FIELDNAME(InputRoot.*[idx] ...") and its position within the message.
Some additional questions I have:
- (why) would I need to store the Properties as well?
- if I am right I could either store the CCSID/Encoding together with the data for each part or use the same combination both to write and to read the data (with the performance implications mentioned by kimbert). Would it be an alternative to use the EmbeddedBitStream option for those parts following the MQMD? |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jan 13, 2009 4:09 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
- (why) would I need to store the Properties as well?
|
Properties cannot be serialized. You are already going to store the CCSID/Encoding. You should certainly check whether you are storing anything else in Properties which is needed later in the flow.
| Quote: |
| - if I am right I could either store the CCSID/Encoding together with the data for each part or use the same combination both to write and to read the data (with the performance implications mentioned by kimbert). |
You *could* standardise on, say, UTF16 for all bitstreams in your database. But I cannot see any benefit in doing that. Remember that code page conversions do not always round-trip accurately ( a single Unicode code point can sometimes map to two or more code points in the source code page ).
| Quote: |
| Would it be an alternative to use the EmbeddedBitStream option for those parts following the MQMD? |
Not sure what you are suggesting here. |
|
| Back to top |
|
|
| haeberm |
| Posted: Tue Jan 13, 2009 4:50 am Post subject: |
|
|
Newbie
Joined: 23 Dec 2008
Posts: 9
|
| Quote: |
| Not sure what you are suggesting here. |
Maybe I got something completely wrong, I just read this in the InfoCenter (ASBITSTREAM reference):
| Quote: |
EmbeddedBitStream, in which not only is the algorithm that generates the bit stream is the same as the algorithm that is used by an output node, but also the
* Encoding
* CCSID
* Message set
* Message type
* Message format
are determined, if not explicitly specified, in the same way as the output node. That is, they are determined by searching the previous siblings of FieldReference on the assumption that they represent headers. |
I thought that using this option the CCSID/Encoding whould always be "right". Clearly, for MQMD I'd have to specify it. |
|
| Back to top |
|
|
|
|
