ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Tagged MRM question for a newbie

Post new topic  Reply to topic Goto page 1, 2  Next
 Tagged MRM question for a newbie « View previous topic :: View next topic » 
Author Message
dazza
PostPosted: Tue Apr 18, 2006 4:41 am    Post subject: Tagged MRM question for a newbie Reply with quote

Newbie

Joined: 18 Apr 2006
Posts: 9

Hi all,

I am a relative newbie to WBI v5 and I could badly use a bit of help.

Here is the scenario:

I am having to process incoming data which has the following format:

xxxxAAAAAA<CR><LF>BBB<CR><LF>CCCCCCCCCC<CR><LF> etc.

where xxxx is the number of times the data fields repeat. AAA, BBB,CC etc represents a data line. Each line of data is of variable length up to a maximum of 250 characters.

From this input format I need to create an output format whereby all <CR><LF>'s are removed and each of the output lines is padded with spaces to a length of 256 characters.

e.g.
AAAAAA<250 spaces>BBB<253 spaces>CCCCCCCCCC<246 spaces>

What do you propose would be the easiest solution to this problem?

Thanks in advance for any assistance.

Regards,
Daz.
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Tue Apr 18, 2006 5:06 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

So, you've got variable length delimited data with a header block indicating the number of records in the file.

You need to replace this with fixed length, non-delimited data. Do you still need the header block?
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
dazza
PostPosted: Tue Apr 18, 2006 5:15 am    Post subject: Reply with quote

Newbie

Joined: 18 Apr 2006
Posts: 9

Hi there. Thanks for the reply.

Yes you are perfectly correct in what you said regarding my requirements.

Would it make it a lot easier if I got rid of the 4 digit repeat count field from the incoming message, leaving just the data itself
i.e.
AAAA<CR><LF>BBB<CR><LF>CCC<CR><LF>etc?

In terms of my data requirements, there is no actual use for the 4 digit repeat count field.
Back to top
View user's profile Send private message
kimbert
PostPosted: Tue Apr 18, 2006 5:28 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

I would use a structure like this:

Code:

Message  (  Data Element Separation = Variable Length Elements Delimited, Delimiter=<CR><LF>  )
    RepeatCount    (  Length = 4  )
    Data  (  minOccurs=0 maxOccurs=-1  )

(Variable Length Elements Delimited is designed for exactly this situation, where all elements have a delimiter except for those which have a length )

For the output side, create a similar structure in which the Data Element Separation for the message is 'Fixed Length', and the 'Data' element has a length of 256.
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Tue Apr 18, 2006 5:40 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

If you don't need it, don't send it - right? Since you're working with delimited data, you can't use CWF anyway, so having repeat counts doesn't help. (Unless CWF can handle "terminated by end of record", but I don't think it can.)

I'm playing around with this a bit to get it right, but you basically just need to do two TDS messages within the same message definition file- one with the delimiter and one without.

The padding will occur on output as a natural consequence of the model.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
kimbert
PostPosted: Tue Apr 18, 2006 5:47 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

CWF can handle 'Repeat to end of bitstream'. However, CWF could not parse this message, because it can only handle strings which are terminated by a NUL.

The repeat count is a non-issue really. It doesn't complicate the model too much, so if it's easier to leave it in, that's fine.
Back to top
View user's profile Send private message
dazza
PostPosted: Wed Apr 19, 2006 2:37 am    Post subject: Reply with quote

Newbie

Joined: 18 Apr 2006
Posts: 9

Hi all.

I am still struggling with this. I am new to message set creation - in the past all the message sets I have done have been via relatively simple Cobol

Copybooks.

I am sorry if this is a long post, but I thought I better explain everything I have done in order for somebody to find out where I have gone wrong. As I am

struggling to get this to work, I decided to simply things slightly by removing the 4 byte repeat_count field at the start of my data.

So, from scratch, this is what I have done:

Resource Navigator -> Right Click -> New -> Message Set Project. Project name : Delim_Test [Next] Message Set Name : Delim_MsgSet [Next]
Tick Tagged / Delimited String Format Name : TDS1

Click on message set name -> Properties Heirarchy -> Runtime Parser : MRM
Physical properties - TDS1
Message Standard : UNKNOWN
Delimiter : <CR><LF>
All the restr are blank and/or default values.

Kimbert - In your reply you mentioned : Data Element Separation = Variable Length Elements Delimited. In my version 5.0.4 toolkit, I cannot see anything

called Variable Length Elements Delimited.


Then New -> Message Definition File. Select Create A New Message definition file [Next] and enter delim_test.txt as filename [Finish]


In Outline window, right click on Message and select Add Message
Message Name : Delim_Testing
Right Clcik -> Add Local Element. Field called DATA

So, now I have a message
Delim_Testing (ComplexType1)
DATA (xsd : String)


As suggested by Kimbert, I click on the DATA field and click on Properties and set.
Logical properties -> Local element -> Occurrences -> Max occurrences to -1


I now have two messages in the Task log window - 1 warning and 1 error.

1 - warning - Element 'DATA' has default TDS length of zero.

2 - Error - Element 'DATA' must have a value of MaxOccurs to comply with its enclosing structure with Data Element Separation of "Fixed Length". Physical

format 'TDS1'


To get rid of the warning I change Physical properties -> TDS1 -> Local element -> Physical Representation -> Length to 256 (which is the max the incoming

field will ever be). Is this correct? I am guessing this is not what I should do because the DATA field should be delimited by the <CR><LF>.

I am guessing the error is because MaxOccurs is set to -1 and the toolkit doesn't realise this is a <CR><LF> delimited field. I am also guessing this is

going back to what I said earlier regarding not being able to see any kind of "Data Element Separation = Variable Length Elements Delimited." setting/value.


Any help you be greatfully appreciated.

Thanks in advanced.
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Wed Apr 19, 2006 2:40 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

DataElementSeparation would be set on the TDS properties of the Complex Type of your Message.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
kimbert
PostPosted: Wed Apr 19, 2006 2:54 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Handy tip when working with message sets:
Quote:
As suggested by Kimbert, I click on the DATA field and click on Properties...
Good. Leave the editor view on the Properties tab permanently. The default spreadsheet view just duplicates what the Outline view (bottom left) shows you.

As Jeff says, Data Element Separation is a TDS property of a Complex Type. In your notation, you need to set
Physical Properties -> Complex Type -> Data Element Separation -> Variable Length Elements Delimited

The warning and the error will disappear once you set the Data Element Separation. You should set the length of 'Data' back to zero - otherwise the parser will think that it is a fixed-length element, and ignore the delimiter.
Back to top
View user's profile Send private message
dazza
PostPosted: Wed Apr 19, 2006 3:32 am    Post subject: Reply with quote

Newbie

Joined: 18 Apr 2006
Posts: 9

Thanks for the help. Much appreciated.

I am sorry to be a pain in the neck. I still have one error which now reads:

Element 'DATA' must have a value for maxOccurs to comply with its enclosing structure with Data Element Separation of 'VariableLengthElementsDelimited'. Physical format: 'TDS1'.

As the both of you suggested, I reset the DATA element length back to zero.
I set the max occurs value to -1 under Logical Propierties -> Local Elements -> Occurrences -> Max Occurs.


Also, thinking ahead to once I have successfully got the input and output MRM formats completed, do I have to use a mapping node or any eSQL in a compute node to parse from the input to the output?

Thanks once again.
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Wed Apr 19, 2006 3:56 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

You will have to use at least some ESQL somewhere - in a mapping node or not as you choose.

But here's the thing. You only need to change one thing. Your logical message structure in both messages is exactly the same. So all you need to do is instruct the parser that it should use a different physical definition to write the logical tree back to a bitstream.

You just have to change the Properties to indicate which Message Type to use.
Code:
 Call CopyEntireMessage();
Set OutputRoot.Properties.MessageType='MessageB';

_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
kimbert
PostPosted: Wed Apr 19, 2006 3:59 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Aha. That's the problem when I post without actually trying it out

You've got two options.
a) Set maxOccurs to a big number. If it works, great. If not...
b) You need to use embedded groups ( gasp!). A Group has its own Data Element Separation property which applies to all its members.
- Put the first element (RepeatCount) inside a group which has DES of 'Fixed Length'.
- Put Data inside a group which has DES of 'All Elements Delimited'.
- Set the Repeating Element Delimiter on Data to <CR><LF>

So your message model will look like this:

Code:
Message
    Group (DES=Fixed Length)
        RepeatCount (Length=4)
    Group (DES=All Elements Delimited)
        Data (Repeating Element Delimiter=<CR><LF> maxOccurs=-1)
Back to top
View user's profile Send private message
dazza
PostPosted: Thu Apr 20, 2006 1:52 am    Post subject: Reply with quote

Newbie

Joined: 18 Apr 2006
Posts: 9

I had production issues to deal with yesterday afternoon so I have only just got around to picking up this again.

Thanks to all the help you have provided me, I got the input message working. To get it to work I changed the max occurs value for the DATA element from -1 to 65535.

I created a simple flow which contained a trace node. This is what the MRM segment of the trace file generated:

Code:

  (0x01000021):MRM        = (
    (0x0300000B):REPEAT = 5
    (0x0300000B):DATA   = 'AAAA'
    (0x0300000B):DATA   = 'BBBBBBB'
    (0x0300000B):DATA   = 'CCCCCCCCCCCCCCCCCCCCCCC'
    (0x0300000B):DATA   = 'DDDDDDDDDDDDDDDDDD'
    (0x0300000B):DATA   = 'EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE'
  )


Is this what you would expect the trace to look like? Also, I am assuming what the above trace is showing is the DATA element is now in an "array" form i.e. InputRoot.MRM.DATA[1], InputRoot.MRM.DATA[2] etc.


I am now about to create the output format, which just contains a DATA element i.e. there is no REPEAT element on the output message.

Just to clarify some of the things mentioned above, is this all I need to do:

Complex Type -
Physical properties -> TDS1 -> Complex Type -> Field Identification -> DES set to fixed length

DATA element -
Leave max occurs at 65535.
Set Physical Properties -> TDS1 -> Local element -> Physical Representation -> Length to 256.

Is there anything else I need to do?

Once again, thanks for all the help so far.
Back to top
View user's profile Send private message
kimbert
PostPosted: Thu Apr 20, 2006 2:21 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Quote:
Is this what you would expect the trace to look like?
Yes - that's exactly what I would expect.
Quote:
Also, I am assuming what the above trace is showing is the DATA element is now in an "array" form i.e. InputRoot.MRM.DATA[1], InputRoot.MRM.DATA[2] etc.
Correct again.

Quote:
DATA element -
Leave max occurs at 65535.
1) Your output message definition is different from your input message definition, so you will need to explicitly set maxOccurs on the DATA element in the output message definition.
2) If you set maxOccurs to 65535 when writing a fixed-length field, it will write 65535 occurrences of the field. Probably not what you want! Set minOccurs to 0 and maxOccurs to -1.
Back to top
View user's profile Send private message
dazza
PostPosted: Thu Apr 20, 2006 4:44 am    Post subject: Reply with quote

Newbie

Joined: 18 Apr 2006
Posts: 9

Arrrggghhh!!! This is driving me mad!!!

I tried to change the maxoccur value to -1 but that gave me the following error:

Element 'DATA' must have a value of MaxOccurs to comply with its enclosing structure with Data Element Separation of "Fixed Length". Physical format 'TDS1'


I remembered something you mentioned yesterday about using groups.

So, I created a new output message. Under this new message I created a group and then assigned the DATA element to the group.

I set the group properties as follows:
DES : Fixed Length
Min Occur : 1
Max occur : -1

I set the values of the DATA element to:
Physical Type : Characters (the default which cannot be changed)
Length : 256
Justification : Left Justify
Padding character : SPACE
Min occurs : 1
Max occurs : 1


I have the following code a compute node:
Code:

CREATE FUNCTION Main() RETURNS BOOLEAN
   BEGIN
      CALL CopyMessageHeaders();
            
      SET OutputRoot.Properties.MessageSet = 'ASDEDI_SeverfieldReeve_Out_MsgSet';
      SET OutputRoot.Properties.MessageType = 'SeverfieldReeveOut_Mess';
      
     DECLARE loop_no     INTEGER;
     DECLARE total_loops INTEGER;

      SET loop_no = 1;
      SET total_loops = InputRoot.MRM.REPEAT;
      
      WHILE loop_no <= total_loops DO
        SET OutputRoot.MRM.DATA[loop_no] = InputRoot.MRM.DATA[loop_no];
        SET loop_no = loop_no + 1;
      END WHILE;
      
      RETURN TRUE;
   END;


A trace node AFTER the compute node shows the following:

Code:

(0x01000000):Properties = (
    (0x03000000):MessageSet      = 'ASDEDI_SeverfieldReeve_Out_MsgSet'
    (0x03000000):MessageType     = 'SeverfieldReeveOut_Mess'
    (0x03000000):MessageFormat   = 'TDS1'
    (0x03000000):Encoding        = 546
    (0x03000000):CodedCharSetId  = 437
    (0x03000000):Transactional   = TRUE
    (0x03000000):Persistence     = FALSE
    (0x03000000):CreationTime    = GMTTIMESTAMP '2006-04-20 12:34:35.100'
    (0x03000000):ExpirationTime  = -1
    (0x03000000):Priority        = 0
    (0x03000000):ReplyIdentifier = X'000000000000000000000000000000000000000000000000'
    (0x03000000):ReplyProtocol   = 'MQ'
    (0x03000000):Topic           = NULL
  )
  (0x01000000):MQMD       = (
    (0x03000000):SourceQueue      = 'QA.GMT.MQSI.ASDEDI.SEVERFIELD.IN'
    (0x03000000):Transactional    = TRUE
    (0x03000000):Encoding         = 546
    (0x03000000):CodedCharSetId   = 437
    (0x03000000):Format           = '        '
    (0x03000000):Version          = 2
    (0x03000000):Report           = 0
    (0x03000000):MsgType          = 8
    (0x03000000):Expiry           = -1
    (0x03000000):Feedback         = 0
    (0x03000000):Priority         = 0
    (0x03000000):Persistence      = 0
    (0x03000000):MsgId            = X'414d512052544d51424b4432202020204444bb17202a5a0d'
    (0x03000000):CorrelId         = X'000000000000000000000000000000000000000000000000'
    (0x03000000):BackoutCount     = 0
    (0x03000000):ReplyToQ         = '                                                '
    (0x03000000):ReplyToQMgr      = 'RTMQBKD2                                        '
    (0x03000000):UserIdentifier   = 'mqm         '
    (0x03000000):AccountingToken  = X'16010515000000d3552b4e302d046895ba775b4848000000000000000000000b'
    (0x03000000):ApplIdentityData = '                                '
    (0x03000000):PutApplType      = 11
    (0x03000000):PutApplName      = '                            '
    (0x03000000):PutDate          = DATE '2006-04-20'
    (0x03000000):PutTime          = GMTTIME '12:34:35.100'
    (0x03000000):ApplOriginData   = '    '
    (0x03000000):GroupId          = X'000000000000000000000000000000000000000000000000'
    (0x03000000):MsgSeqNumber     = 1
    (0x03000000):Offset           = 0
    (0x03000000):MsgFlags         = 0
    (0x03000000):OriginalLength   = -1
  )
  (0x01000021):MRM        = (
    (0x0300000B):DATA = 'AAAA'
    (0x0300000B):DATA = 'BBBBBBB'
    (0x0300000B):DATA = 'CCCCCCCCCCCCCCCCCCCCCCC'
    (0x0300000B):DATA = 'DDDDDDDDDDDDDDDDDD'
    (0x0300000B):DATA = 'EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE'
  )


In the trace file, should I see all the DATA elements padded out to 256 characters?


The error entry in the AIX syslog.user shows the following:

Code:

(WBI_RTMQBKD2_BROKER.exec_UK_CEI)[7456]BIP2628E: Exception condition detected on input node 'CCI_EDI_SeverfieldReeve_UK.MQInput'. : WBI_RTMQBKD2_BROKER.374ac3e3-0601-0000-0080875c038b9d78:/build/S500_P/src/DataFlowEngine/ImbMqInputNode.cpp: 2083:ImbMqInputNode::readQueue:ComIbmMQInputNode:CCI_EDI_SeverfieldReeve_UK#FCMComposite_1_1

(WBI_RTMQBKD2_BROKER.exec_UK_CEI)[7456]BIP2230E: Error detected whilst processing a message in node 'CCI_EDI_SeverfieldReeve_UK.MQOutput'. : WBI_RTMQBKD2_BROKER.374ac3e3-0601-0000-0080-875c038b9d78: /build/S500_P/src/DataFlowEngine/ImbMqOutputNode.cpp: 811: ImbMqOutputNode::evaluate: ComIbmMQOutputNode: CCI_EDI_SeverfieldReeve_UK#FCMComposite_1_2

(WBI_RTMQBKD2_BROKER.exec_UK_CEI)[7456]BIP5286E: Message Translation Interface Writing Errors have occurred:  : WBI_RTMQBKD2_BROKER.374ac3e3-0601-0000-0080-875c038b9d78: /build/S500_P/src/MTI/MTIforBroker/MtiImbParser2/MtiImbParser.cpp: 1231: MtiImbParser::refreshBitStreamFromElements - 9 par: ComIbmMQInputNode: CCI_EDI_SeverfieldReeve_UK#FCMComposite_1_1

(WBI_RTMQBKD2_BROKER.exec_UK_CEI)[7456]BIP5447S: Tagged/Delimited String Format (TDS) writing error while writing message 'SeverfieldReeveOut_Mess' : WBI_RTMQBKD2_BROKER.374ac3e3-0601-0000-0080-875c038b9d78: /build/S500_P/src/cpi/pwf/tds/tdswriter.cpp: 256: CTDSWriter::write: :

(WBI_RTMQBKD2_BROKER.exec_UK_CEI)[7456]BIP5452E: Mandatory Element 51^DATA has no value or default value set. : WBI_RTMQBKD2_BROKER.374ac3e3-0601-0000-0080-875c038b9d78: /build/S500_P/src/cpi/pwf/tds/tdswritecontext.cpp: 2231: CTDSWriterContext::SubstituteDefaultValue PWFData & Data, ELogicalType FieldType, int TagIndex): :


which suggests that even though the trace is showing data in the DATA elements, it isn't actually getting assigned to the output message when passed to the MQOutput node.

Is there something I have done wrong in the eSQL code? Or is it more a problem with my output message set?

Cheers.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Tagged MRM question for a newbie
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.