| Author |
Message
|
| marko.pitkanen |
 Posted: Mon Dec 03, 2012 9:21 am Post subject: Scaling one-way inbound (from SAP to broke) pattern for IDoc |
 |
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
Hi,
I did some researching, but perhaps not enough so perhaps someone can point me a to the thread this has been discussed earlier.
Situation:
- WMB 7.0.0.4 on Linux
- IBM WebSphere Adapter for SAP Software 7.0.0.4_IF04
- Ensure assured-once delivery on
- Number of listeners 1
- Ale update status on
- tRFC is used
Depending on the SAP's way of packaging
- Send ALEAUD per packet on
- Ignore IDoc packet errors on
We have been trying to speed up generic iDoc passthrough processing by adding additional instances for the flow.
At the same EG identical adapter is listening SAP just with one instance and different RFC Program ID for the IDocs where they want FIFO.
At the SAP's side they are occurring quite a lot "connection lost for RFC partner" exceptions and ALEAUD messages are every now and then incorrect. And sometimes SAP thinks they could send x iDocs successfully, but from adapter flow's outbound queue we can find x+n iDocs.
At broker' side I can see " Status for Idoc is invalid", "failed to process an event called 'SapGenericIDocObject'", ".ArrayIndexOutOfBoundsException: No such row 14; possible: [0--1]" exceptions for example.
And if they reprocess iDocs in RFC -failure, only control record comes to queue from adapter for some of them.
Hopefully I'm doing something simple incorrectly, and you can enlighten me the pain point.
What are the correct ways to scale the usage of this pattern?
--
Marko
Last edited by marko.pitkanen on Mon Dec 03, 2012 9:54 am; edited 1 time in total |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Mon Dec 03, 2012 9:32 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
You might consider front-ending your SAP message flow with a separate SAP message flow that acts as a gateway.
Part of the problem with SAP is timing. The more responsive you can make WMB processing, the less ALE failures you will notice.
Seems like your current implementation gets many of these timing issues.
| Quote: |
| SAP returns only a return code and a null object to the caller. If no exceptions are raised, the outbound transaction is considered successful. The success of the transaction can be verified by inspecting the IDocs that have been generated in SAP. |
http://www.ibm.com/developerworks/websphere/library/techarticles/1101_batey/batey.html
Success or failure cannot be determined at transaction time and must be determined by looking up the info after the transaction has been processed. Requeue failed transactions.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Mon Dec 03, 2012 9:56 am Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
Sorry,
Direction was stated incorrectly, but changed now at the original post.
--
Marko |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Dec 03, 2012 9:59 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Mon Dec 03, 2012 11:32 am Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
Thanks lancelotlinc,
I don't perhaps completely understand the concept of SAP gateway message flow. I need to do more investigation. I thought that this simple passthrough message flow that receives all kind of IDocs with SapGenericIDocObjec and passes particular IDoc bytestream data to the downstream message flow for further processing acts as a gateway flow.
One thing that I didn't mention is that SAP and adapter are not located in the same site. So there might be latency even at the communication layer.
--
Marko |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Dec 03, 2012 11:41 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Here are some general resources that may help:
http://www.redbooks.ibm.com/redpapers/pdfs/redp4644.pdf
http://www-01.ibm.com/support/docview.wss?uid=swg27018476&aid=1
| Quote: |
| Use wait parameter before calling BAPI commit parameter can be used to control whether the SAP adapter waits for SAP to commit the updates synchronously, or issues a commit and returns while the SAP commit happens asynchronously. |
| Quote: |
Performance considerations: The following parameters may be used for performance tuning:
- Number of listeners on the inbound SAP adapter controls the
maximum number of concurrent calls coming from SAP
system to SAPInput node
- Listener tuning is applicable to inbound interfaces only
- Additional instances may also be configured on the SAPInput
node and/or the BAR file itself to enhance the throughput |
You're going to have to experiment with your message flows to get the desired outcome.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Mon Dec 03, 2012 12:30 pm Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
Really valuable readings.
Thanks lancelotlinc!
--
Marko |
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Mon Dec 10, 2012 11:48 am Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
As a status update: we will update our adapter to the latest recommended fix pack level and do the same testing. It seems that there are some improvements at the area of ALEAUD handling for example.
--
Marko |
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Fri Jan 11, 2013 5:27 am Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
As a status update for this thread: we upgraded broker to version 7.0.0.5 and jco to 3.0.9, but it didn't help a much. We will rise a PMR next week.
--
marko |
|
| Back to top |
|
|
| jhosie |
| Posted: Mon Jan 14, 2013 2:41 am Post subject: |
|
|
Apprentice
Joined: 12 May 2005
Posts: 28
|
Marko,
A few things to note....
Increasing the additional instances of the flow is the correct approach to scale this pattern. For each instance of the flow, the broker will create another listener connection to SAP (effectively, increasing the number of listeners property) as well as spawning another message flow thread to handle the IDOCs in parallel.
For this to work, you will probably need to make a corresponding change on SAP to allow multiple listener connections. By default, each RFC Destination only allows 1 listener. To adjust this, you need to open the SAP GUI and:
1 run transaction SMQS
2 tick the RFCSERVER you want to change
3 hit the "Registration" button at the top
4 change the MAXCONN value and press the green tick.
(or ask your SAP admin team).
That should help to create parallel connections to SAP and to process IDOCs in parallel through the gateway flow.
From your original post, however, it does not seem that concurrency is the issue. The exception text that you see on the broker side suggests something functional is going wrong. Could it be that the IDOC definitions are out of sync? Maybe they have been changed on the SAP side since you discovered them on the the broker toolkit? Are there any other errors that might shed some light on this?
Finally, you say that the adapter is not on the same site as SAP. You are correct that this could introduce some latency. If you can you should test this, but you might find that running the gateway on the same LAN as the SAP system, and then crossing the WAN using MQ is more optimal. The problem with crossing the WAN on the sap connection is that the connection remains idle - no data is transferred - while the message flow is busy. This is a consequence of the transactional coordination between the message flow and SAP. |
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Wed Apr 10, 2013 7:12 am Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
As an update for this thread. I have been informed that support have been able to reproduce the phenomenon at their environment and are investigating the root cause.
--
Marko |
|
| Back to top |
|
|
|
|
