| Author |
Message
|
| whydieanut |
 Posted: Fri Dec 09, 2011 1:31 am Post subject: SAP Adapter Design Pattern for ALE |
 |
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
Hi all.
I am looking for suggestions on how to efficiently use the SAP Adapter while in ALE mode.
I have to send multiple types of IDOCs, coming from different applications to an SAP server.
I only have a single RFC destination on the SAP server. Given that:
>> Is it advisable to use a single ALE Out Adapter to send data for ALL types of IDOCs, to SAP?
(Eg. IDOC1, IDOC2, IDOC3, IDOC4, IDOC5, IDOC6 sent to Adapter1)
Then I have 1 flow per IDOC type, which will all forward the IDOCs to a single Common Flow with the Adapter.
| Code: |
Input1 -> Flow to process IDOC1 -> Intermediate Queue -> Common Flow with Adapter1 -> SAP
Input2 -> Flow to process IDOC2 -> Intermediate Queue -> Common Flow with Adapter1 -> SAP
.
.
Input6 -> Flow to process IDOC6 -> Intermediate Queue -> Common Flow with Adapter1 -> SAP
|
>> Or is it better to club together the different IDOC types into different Out Adapters?
(Eg. IDOC1, IDOC2 and IDOC3 sent to Adapter1; IDOC4, IDOC5, IDOC6 sent to Adapter2)
Then I have 1 flow per IDOC type, which will forward their respective IDOCs to either of 2 Common flows (with the Adapter) depending on the type of IDOC.
| Code: |
Input1 -> Flow to process IDOC1 -> Intermediate Queue 1 -> Common Flow with Adapter1 -> SAP
Input2 -> Flow to process IDOC2 -> Intermediate Queue 1 -> Common Flow with Adapter1 -> SAP
Input3 -> Flow to process IDOC3 -> Intermediate Queue 1 -> Common Flow with Adapter1 -> SAP
Input4 -> Flow to process IDOC4 -> Intermediate Queue 2 -> Common Flow with Adapter2 -> SAP
Input5 -> Flow to process IDOC5 -> Intermediate Queue 2 -> Common Flow with Adapter2 -> SAP
Input6 -> Flow to process IDOC6 -> Intermediate Queue 2 -> Common Flow with Adapter2 -> SAP
|
|
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Fri Dec 09, 2011 7:51 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And the answer will probably be: it depends on whether you need different error handling for case 1 or case 2... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| inMo |
| Posted: Fri Dec 09, 2011 6:07 pm Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
| Quote: |
| I only have a single RFC destination on the SAP server... |
Given the single RFC destination, what would be the point of splitting the "common" flow into 2?
If you are willing to consider clarity of purpose as a priority for your design, perhaps a single RFC DEST would be best served by a single partnered adapter? Operations might appreciate that kind of thinking.
All that said, certain behavioral reqs (such as the different error handling consideration introduced be fjb_saper) might steer you away from a single adapter. The key idea is that something would need to be different other then the IDOC #, but thus far nothing like that has been introduced. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Sun Dec 11, 2011 11:59 pm Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
| fjb_saper wrote: |
| And the answer will probably be: it depends on whether you need different error handling for case 1 or case 2... |
The Error Handling requirements are the same.
| inMo wrote: |
| Given the single RFC destination, what would be the point of splitting the "common" flow into 2? |
I wanted to know if there is any performance impact of having a single Adapter handle all data vs having IDOCs split up to different Adapters, based on perhaps the load of messages for each type of IDOC so that say 10,000 messages are split up between 2 Adapters, each handling just 5000 messages.
Also I wanted to split the IDOCs between adapters, so that in case of any new IDOCs being introduced later on, I might only have to update specific Adapters, thus keeping other Adapters, (at their common flows) from being impacted in any way. |
|
| Back to top |
|
|
| inMo |
| Posted: Mon Dec 12, 2011 4:52 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
Let me answer in reverse order of your questions. Your second question is about system management. Unless there is a serious benefit to justify added complexity, I suggest you stick with simplicity. It is ultimately easier to direct operations (aka someone who knows nothing about YOUR system/design) to manage THE adapter vs adapter 1 of 2 (which ultimately means 1 of n in the long run). Regardless of the number of adapters, you will need a procedure to handle new idocs. The effect of a flawed procedure applied against adapter 2 still impacts 50% of all idocs. The argument that 50% were "protected" from a flawed procedure still translates to "we broke the system"
question 1 was about performance. I'd schedule a meeting with your SAP admin team. The conversation should be around the idea of multiple resources using the same RFC dest. Does it cause repeated handshakes? Can both resources simultaneously maintain an open handle? We can make an educated guess that 1 adapter gaurantees that. Therefore, IMHO, the starting position is there is not yet justification to move away from the simplest model: 1 adapter. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Tue Dec 13, 2011 10:33 pm Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
Thanks for the input.
Will talk to the SAP guys about it.
So as of now, there is no compelling enough reason to have multiple Adapters. This is for Outbound.
Coming to Inbound Adapters:
I don't really have an option of having multiple Inbound Adapters using the same Program ID.
I've tried creating 2 Adapters, each with different sets of IDOCs, but then the Adapters pick up IDOCs randomly.
Meaning, an IDOC I was expecting to be read by Adapter 1 was being read by Adapter 2, where it failed because this Adapter wasn't configured for that type of IDOC.
So for Inbound there is really no option but to use a single Adapter.
But then there is an option of using the Adapter in Passthrough mode for Inbound.
Is there any good reason to use the Passthrough Adapter for Inbound?
For one, it lets me keep the Adapter unaffected in case any new IDOCs are added later on.
But there is some issue with Broker V7 Fix Pack 1 when using the Inbound Adapter in Passthrough mode. This is fixed in Fix Pack 3. But then normal Outbound Adapter has an issue in FP3.
So the question now is, do we need the Passthrough mode?
If there is not enough benefit in using it, then FP1 works fine for both Inbound and Outbound and we can proceed with it.
But if Passthrough is better than using a single normal Inbound Adapter, then we have to go with FP3, for which we have to raise a PMR because of the Outbound Adapter issue in it.
Any advice? |
|
| Back to top |
|
|
| inMo |
| Posted: Wed Dec 14, 2011 4:57 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
For outbound (adapter perspective) I happen to like the way you phrased your summary: as of now ... That still allows for more information to be gathered.
For Inbound (adapter perspective) it appears you've done your homework. Is SAP team a working partner here or are they more or less a black box? If they will work with you, you can explore unique program ids for each adapter, but that may not be desirable either. Registering 2 adapters with the same program id is effectively telling SAP "hey - I've got 2 identical processes out here - do you mind load balancing for me?" |
|
| Back to top |
|
|
| whydieanut |
| Posted: Thu Dec 15, 2011 2:19 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
I spoke to the SAP guys, and it seems having multiple RFC destinations to a single system is not recommended, and I think not necessary at all.
There is this other bit of info, that I just remembered.
I am talking about Inbound Adapters here.
Having a single Inbound Adapter configured for multiple types of IDOCs has been tested and it works fine.
As long as the types of IDOCs are limited in number or within certain size limits.
I tried using a single Adapter with 2 types of IDOCs and it worked.
But then as soon as I added another IDOC type, one with a huge number of fields (So large that the toolkit hanged while updating the Adapter using the wizard), I now had a equally HUGE Message Set that the Adapter needed, the flow deployment timed out on my local machine.
Will test multiple types of such IDOCs having large number of fields once again though.
My question is, when there are a large number of IDOCs or Large IDOC structures, the associated Message Set is bound to be huge in size, and cause problems during deployment.
Is there any work around for this problem, or is the only solution to have multiple Adapters (using different Program Ids)?
This is when the Passthrough mode Adapter seems useful.
In it's case, the Message Set that it requires is simple with only 1 Message type defined (SAP Generic IDOC).
I can then filter the IDOCs depending on their type and forward them as BLOBs to separate flows, where they will be parsed into IDOCs using separate smaller Message Sets with only 1 type of message defined.
So I don't have a single flow with a huge Message Set.
Is there any other way of looking at this? |
|
| Back to top |
|
|
| inMo |
| Posted: Thu Dec 15, 2011 6:06 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
I recognize that I am far removed from the details, but on the surface, I don't generally see the point in having a very intelligent/all knowing inbound adapter. I think you've discovered that the all-knowing part comes with an all-available resource requirement. And for what value? I think all you need to do is take a peek at the idoc and pass it off to the process that is specifically designed to deal with that particular idoc. This has multiple advantages, not the least of which is spreading resource requirements and coding challenges across smaller, more manageable components.
One other advantage I can't help but mention is maintenance. In your investigation, you discovered a scenario where adding the specific knowledge of an extra-ordinary idoc caused the "common" flow to become unusable. By keeping the common flow generic and simple in nature, you will eliminate 95% of maintenance. In other words, the extra-ordinary idoc would pose challenges to ONLY the specific process designed to work with it, and not force the common flow to deal with it. This translates to seperation of concern, and those processes not concerned with the extra ordinary idoc will continue to run just as they always have.
| Quote: |
| This is when the Passthrough mode Adapter seems useful. |
That is because it is.
| Quote: |
| Is there any other way of looking at this? |
Yes. If you leverage passthrough and keep the design simple, you will end up with a model that allows a single flow to handle any and all inbound to WMB idocs. With that flow, you will infact be able to have 1 to n threads running. Once you go beyond a single thread, you will automatically trigger SAP's load balancing feature. In other words, to increase overall throughput, you won't have to re-invent the design. You may simply increase the total threads running and it is a very robust solution (assuming you have no sequential processing reqs  ) |
|
| Back to top |
|
|
| whydieanut |
| Posted: Thu Dec 15, 2011 11:31 pm Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
Thanks a lot for the feedback!
I will most probably go ahead with the Passthrough Mode Inbound Adapter.
So here's the high level design I was looking at:
Main Inbound Flow:
| Code: |
-> [IDOC1 Queue]
SAP -> [Single Passthrough Inbound Adapter] : [Single Common Inbound Flow] -> [IDOC2 Queue]
-> [IDOC3 Queue]
|
Individual IDOC Processing Flow:
| Code: |
:[Msg Type for System1] -> [SYS1]
[IDOC1 Queue] -> [Process IDOC1 Flow]:[Output Msg Set1]:[Msg Type for System2] -> [SYS2]
:[Msg Type for System3] -> [SYS3]
[IDOC2 Queue] -> [Process IDOC2 Flow]:[Output Msg Set2]:[Msg Type for System2] -> [SYS2]
:[Msg Type for System4] -> [SYS4]
|
Anything I am missing...? |
|
| Back to top |
|
|
| whydieanut |
| Posted: Tue Aug 21, 2012 10:40 pm Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
Ok, Back to an old thread...
Having looked at how requirements are progressing, I am having trouble with the approach taken for the Outbound Adapters...
What I have right now is as follows:
I have multiple Inbound Flows flows each producing 1 of the 3 types of IDocs and putting them into an Intermediate Queue.
I have a common Outbound Flow with a a single Outbound ALE Adapter configured for these 3 types of IDocs.
This flow picks up IDocs put into the Intermediate Queue (generated by the earlier flows)
| Code: |
Flow1 to create IDOC1 ->
Flow2 to create IDOC2 -> Intermediate Queue -> Common Flow with Adapter -> SAP
Flow3 to create IDOC3 ->
Flow4 to create IDOC1 ->
|
Now what's happening is that the IDoc structures are being changed frequently and I am needed to update the Common Adapter for every change in any of the IDocs. This is turning out to be quite a overhead, considering the regression testing needed.
Given this, is it better to just have Individual Adapters for each IDoc type and have corresponding Adapters in the Inbound flows directly, instead of having the Common Outbound flow as below?
| Code: |
Flow1 to create IDOC1 [Adapter1] -> SAP
Flow2 to create IDOC2 [Adapter2] -> SAP
Flow3 to create IDOC3 [Adapter3] -> SAP
Flow4 to create IDOC1 [Adapter1] -> SAP
|
Any advice on this would be greatly appreciated... |
|
| Back to top |
|
|
| mqsiuser |
| Posted: Tue Aug 21, 2012 11:18 pm Post subject: |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
| whydieanut wrote: |
I have a common Outbound Flow with a a single Outbound ALE Adapter configured for these 3 types of IDocs.
This flow picks up IDocs put into the Intermediate Queue (generated by the earlier flows) |
Can you make the common outbound flow independent of the idoc-type (like you did on your inbound flow(s)) ? |
|
| Back to top |
|
|
| whydieanut |
| Posted: Wed Aug 22, 2012 12:59 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
|
| Back to top |
|
|
| mqsiuser |
| Posted: Thu Aug 23, 2012 5:56 am Post subject: |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
Can't you use passthru mode & GenericIDoc on the outbound? You explained that so well as if its obviously the right choice (for inbound and for outbound):
| whydieanut wrote: |
Inbound ALE Adapter in Passthrough mode:
Create a single Adapter of type GenericIDoc.
This would create a very basic Message Set with just one definition - GenericIDocObject.
This parsed message would have a bitstream of the actual IDoc contents which could be passed onto another flow to be parsed.
Thus your Adapter Message Flow need not have the huge Message Set with all IDoc types. Each individual flow for each IDoc type will need a Message Set with definitions for just a single IDoc that is to be processed by the flow. |
If you process files (with WBI-Adapter) you'd have a similiar approach: A message set with just BLOB to send the (to be written) file to the (out-)adapter... so technically that's a viable approach  |
|
| Back to top |
|
|
| whydieanut |
| Posted: Thu Aug 23, 2012 9:36 pm Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
I have tried using the Passthrough Outbound Adapter as well.
As you mentioned, it requires data in the Generic IDoc format wiht 3-4 fields - IDoc Type, IDoc Stream Data (Message BLOB), etc.
It works if I send in the relevant data.
My input message is in a simple XML format which needs to be converted to the IDoc format.
I am currently converting the input XML Message to the XML representation of the IDoc successfully (it gets passed to the normal Outbound Adapter and gets processed in SAP too).
Look at the sample IDoc message in XML format below:
| Code: |
<NS1:SapDebmas07Debmas07Ext xmlns:NS1="http://www.ibm.com/xmlns/prod/websphere/j2ca/sap/sapdebmas07debmas07ext">
<SapDebmas07Debmas07ExtIDocBO>
<SapIDocControlRecord>
<TABNAM>EDI_DC40</TABNAM>
<MANDT>110</MANDT>
<DOCNUM>0000000000203034</DOCNUM>
<DOCREL>702</DOCREL>
<STATUS>30</STATUS>
<DIRECT>1</DIRECT>
<OUTMOD>2</OUTMOD>
<EXPRSS/>
<TEST/>
<IDOCTYP>DEBMAS07</IDOCTYP>
.
.
.
</SapDebmas07Debmas07ExtDataRecord>
</SapDebmas07Debmas07ExtIDocBO>
</NS1:SapDebmas07Debmas07Ext>
|
But to use the Passthrough Adapter, I need to pass data to the Adapter in the Generic IDoc Format.
For this, I need to convert the above XML format message to the SAP ALE IDoc format (using the existing Message Set created while discovering the IDoc Type) which is similar to a CWF message, like below:
(This is what I am unable to achieve with the existing Message Set)
| Code: |
EDI_DC40 1300000000000201089701 3012 DEBMAS07 DEBMAEXT DEBMAS SAPR11 LS R11CLNT130 RCVPORTWMBLS LS_AIWMB ....
|
Then wrap this is the Generic IDoc format inside the IDocStreamData tag like below.
(This is the simple part)
| Code: |
<NS1:SapGenericIDocObject xmlns:NS1="http://www.ibm.com/xmlns/prod/websphere/j2ca/sap/sapgenericidocobject">
<SAPTransactionID>AC150C6908E04E803ACF0610</SAPTransactionID>
<IDocStreamData>
4544495f444334302020313330303030303030303030303230313
03931373031203330313220204445424d415330372020202020202020202020202020
20202020202020204445424d414558542020202020202020202020202020202020202
02020204445424d415320202020202020202020202020202020202020202020202020
202020202020202020202020202020202020534150523131202020204c5320205231
.
.
.
020202020202020202020202020202020202020202020202020202020202020202020
202020202020202020202020202020202020202020202020202020202020202020202
0202020202020202020202020202020202020202020202020202020
</IDocStreamData>
<IDocType>DEBMAEXT</IDocType>
</NS1:SapGenericIDocObject>
|
|
|
| Back to top |
|
|
|
|
