| Author |
Message
|
| fschwarz |
 Posted: Tue Oct 31, 2006 3:02 am Post subject: ESQL POSITION and SUBSTRING |
 |
|
Newbie
Joined: 31 Oct 2006
Posts: 7
|
Hi,
we have an esql node processing a large message (about 28MB) as a BLOB.
We are parsing this message using POSITION to locate a new line in the file and then SUBSTRING to create a new MQ-Message for further processing.
We have two systems (Solaris 9) running same version of Message Broker.
One processes the message in ~1minute.
The other is running for ever.
As of the Message Broker trace I found POSITION and SUBSTRING always running for 10 secs before the action completes (evaluating expression until finished evaluating expression).
This is true for every call to the commands, regardless of the start positions we provide to the functions.
If we run this with a 10 times smaller message, processing of POSITIOn and SUBSTRING is down to 1 sec. So there is definately a relation to the message size.
Using truss to analyze the process activities we found only mem-copy and mem-move operations taking place. Most of those loop throught the memory and copy one byte to 2 bytes. So we assumed there is a character conversion taking place (to UTF-8??).
The program args are identical, both machines run with the same CCSIDs for Broker and MQ (819).
Any ideas? |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Tue Oct 31, 2006 4:31 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Is it the same message, or different messages that are producing different behavior on different machines?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fschwarz |
| Posted: Tue Oct 31, 2006 4:36 am Post subject: |
|
|
Newbie
Joined: 31 Oct 2006
Posts: 7
|
| It is the same message unloaded from queue and provided to both machines. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 31, 2006 4:43 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
And the same flow? (I assume yes, but I do have to ask.)
And there's nothing else running on the machine that runs slow? And you're running the same FixPack levels on both machines?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fschwarz |
| Posted: Tue Oct 31, 2006 4:55 am Post subject: |
|
|
Newbie
Joined: 31 Oct 2006
Posts: 7
|
Hi Jeff,
yes, the systems are identically, we have already reduced the flows and procedures to the basic calculation and seeing the same code behaving differently on two machines.
We already checked:
Broker on same Version (mqsiservice -v)
------------------------------------------------
BIPv500 en US
ucnv Console CCSID 819 dft ucnv CCSID 819
OS 646 ICUW US-ASCII ICUA US-ASCII
BIP8996I: Version: 500
BIP8997I: Product: WebSphere MQ Integrator
BIP8998I: CMVC Level: S500-CSD06
BIP8999I: Build Type: Production
------------------------------------------------
CCSID settings for MQ (runmqsc <qmgr> -> dis qmgr):
------------------------------------------------
AMQ8408: Display Queue Manager details.
DESCR( ) DEADQ(SYSTEM.DEAD.LETTER.QUEUE)
DEFXMITQ( ) CHADEXIT( )
CLWLEXIT( ) CLWLDATA( )
REPOS( ) REPOSNL( )
SSLKEYR(/var/mqm/qmgrs/XNIBR000/ssl/key)
SSLCRLNL( ) SSLCRYP( )
COMMANDQ(SYSTEM.ADMIN.COMMAND.QUEUE) QMNAME(XNIBR000)
CRDATE(2006-04-03) CRTIME(17.30.40)
ALTDATE(2006-10-30) ALTTIME(16.58.1
QMID(XNIBR000_2006-04-03_17.30.40) TRIGINT(999999999)
MAXHANDS(256) MAXUMSGS(10000)
AUTHOREV(DISABLED) INHIBTEV(DISABLED)
LOCALEV(DISABLED) REMOTEEV(DISABLED)
PERFMEV(DISABLED) STRSTPEV(ENABLED)
CHAD(DISABLED) CHADEV(DISABLED)
CLWLLEN(100) MAXMSGL(100000000)
CCSID(819) MAXPRTY(9)
CMDLEVEL(530) PLATFORM(UNIX)
SYNCPT DISTL(YES)
------------------------------------------------
Process arguments (pargs -a -e <ExecGroupPid>)
------------------------------------------------
17514: DataFlowEngine XNIBR000_BRK bb9bc699-0e01-0000-0080-816ca92a858a EXEC_XNIBR002_
argv[0]: DataFlowEngine
argv[1]: XNIBR000_BRK
argv[2]: bb9bc699-0e01-0000-0080-816ca92a858a
argv[3]: EXEC_XNIBR002_BRK_BC_TESTFTP
argv[4]: 0
envp[0]: _=/var/wmqi/brokers/XNIBR000_BRK/bin/mqsistart
envp[1]: LANG=
envp[2]: NLSPATH= --- REMOVED ---
envp[3]: HZ=
envp[4]: MQS_NO_SYNC_SIGNAL_HANDLING=1
envp[5]: PATH=/opt/mqsi/bin:/data/broker/global/bin:/opt/HVBwbitools/ih03/solaris:/opt/HVBwbitools/ih06:/opt/HVBwbitools/ma01/Solaris:/opt/HVBwbitools/mc6d/SunOS:/opt/oracle/product/9.2.0/bin:/opt/oracle/product/9.2.0/ctx/lib:/usr/bin:/usr/local/bin::/usr/local/bin
envp[6]: NLS_LANG=american_america.we8iso8859p1
envp[7]: MQSI_LOCAL_CCSID=819
envp[8]: MQSI_REGISTRY=/var/mqsi
envp[9]: CLASSPATH= --- REMOVED ---
envp[10]: LOGNAME=tuwbi000
envp[11]: MAIL=/var/mail/tuwbi000
envp[12]: HOSTNAME= --- REMOVED ---
envp[13]: PS1=\\h:\\u:\\w$
envp[14]: SHLVL=1
envp[15]: TNS_ADMIN= --- REMOVED ---
envp[16]: LC_CTYPE=iso_8859_1
envp[17]: SHELL=/bin/bash
envp[18]: ORACLE_TERM=xterm
envp[19]: HOSTTYPE=sparc
envp[20]: OSTYPE=solaris2.9
envp[21]: ODBCINI=/data/broker/global/odbc/odbc.ini
envp[22]: HOME=/opt/home/<user>
envp[23]: TERM=vt100
envp[24]: LD_LIBRARY_PATH= --- REMOVED ---
envp[25]: ORACLE_HOME=/opt/oracle/product/9.2.0
envp[26]: PWD= --- REMOVED ---
envp[27]: TZ=MET
envp[28]: MACHTYPE=sparc-sun-solaris2.9
envp[29]: MQS_NO_SYNC_SIGNAL_HANDLING=1
envp[30]: XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt
------------------------------------------------
=> all is identical set or at least set to corresponding values (broker names differ, etc.)
We are currently investigating Broker Properties differences.
Do you know how POSITION is handling BLOB? According to the documentation it is a array structure that is traversed starting with <startposition> and the hex value given is compared to the next position.
What we see is constant POSITION response time (10 secs for this message). We assume a character conversion taking place but have no clue on how to check. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 31, 2006 6:28 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I've no idea how POSITION is actually implemented. I don't know that the people who *do* know that would tell you, either.
In theory, Broker translates all data into Unicode for internal uses. In theory Broker shouldn't be doing that for BLOB data.
What's the actual POSITION statement?
What do you see in debug trace on the machine that "takes forever"? Just the relevant section, thanks.
And, I hope you've opened a PMR.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| wyatt |
| Posted: Tue Oct 31, 2006 6:41 am Post subject: |
|
|
Voyager
Joined: 28 Nov 2004
Posts: 76
|
Was their a resolution on this.
Curious? |
|
| Back to top |
|
|
| fschwarz |
| Posted: Tue Oct 31, 2006 6:52 am Post subject: |
|
|
Newbie
Joined: 31 Oct 2006
Posts: 7
|
The code is this:
-------------------------------------------------
[...]
CREATE COMPUTE MODULE BC_RIB_BatchGattung_SplitAndPropagate_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
DECLARE blbLF CONSTANT BLOB X'0A';
DECLARE intStartPos INT 1;
DECLARE intLFPos INT 0;
DECLARE intSent INT 0;
DECLARE intMsgLen INT LENGTH(InputRoot.BLOB.BLOB);
DECLARE intCopyLen INT 0;
-- read batch data from MQRFH2 header
SET Environment.Variables.BatchID = InputRoot.MQRFH2.ComIbmBC.Batch.ID ;
SET Environment.Variables.START_TS = InputRoot.MQRFH2.ComIbmBC.Batch.START_TS;
SET Environment.XML.Batch.NoData = NULL;
SET Environment.Variables.RecordNo = 0;
SET Environment.Variables.RecordDone = 0;
SET Environment.Variables.RecordFailure = 0;
WHILE (intSent < intMsgLen) DO
-- find next line feed in input blob
SET intLFPos = POSITION(blbLF IN InputRoot.BLOB.BLOB FROM intStartPos);
-- if next Linefeed has been found, send line
IF (intLFPos > 0) THEN
SET intCopyLen = intLFPos - intStartPos;
CALL CopyMessageHeaders();
SET OutputRoot.MQMD.Format = MQFMT_STRING;
-- delete old mqrfh2
SET OutputRoot.MQRFH2 = NULL;
-- create new one
SET OutputRoot.MQRFH2.(MQRFH2.Field)Version = 2;
SET OutputRoot.MQRFH2.(MQRFH2.Field)Format = MQFMT_STRING;
SET OutputRoot.MQRFH2.(MQRFH2.Field)NameValueCCSID = InputRoot.MQMD.CodedCharSetId;
SET OutputRoot.MQRFH2.ComIbmBC.Batch.ID = Environment.Variables.BatchID;
SET OutputRoot.MQRFH2.ComIbmBC.Batch.START_TS = Environment.Variables.START_TS;
SET OutputRoot.BLOB.BLOB = SUBSTRING(InputRoot.BLOB.BLOB FROM intStartPos FOR intCopyLen);
[...]
-------------------------------------------------
The trace output is as follows:
2006-10-31 11:05:06.719143 24 UserTrace BIP2537I: Node 'BC_RIB_BatchGattung.SplitAndPropagate': Executing statement 'SET intLFPos = POSITION(blbLF IN InputRoot.BLOB.BLOB);' at (.BC_RIB_BatchGattung_SplitAndPropagate_Compute.Main, 27.4).
2006-10-31 11:05:06.719176 24 UserTrace BIP2538I: Node 'BC_RIB_BatchGattung.SplitAndPropagate': Evaluating expression 'POSITION(blbLF IN InputRoot.BLOB.BLOB)' at (.BC_RIB_BatchGattung_SplitAndPropagate_Compute.Main, 27.21).
2006-10-31 11:05:06.719203 24 UserTrace BIP2538I: Node 'BC_RIB_BatchGattung.SplitAndPropagate': Evaluating expression 'blbLF' at (.BC_RIB_BatchGattung_SplitAndPropagate_Compute.Main, 27.30).
2006-10-31 11:05:06.719227 24 UserTrace BIP2538I: Node 'BC_RIB_BatchGattung.SplitAndPropagate': Evaluating expression 'InputRoot.BLOB.BLOB' at (.BC_RIB_BatchGattung_SplitAndPropagate_Compute.Main, 27.39).
2006-10-31 11:05:06.719268 24 UserTrace BIP2538I: Node 'BC_RIB_BatchGattung.SplitAndPropagate': Evaluating expression 'intStartPos' at (.BC_RIB_BatchGattung_SplitAndPropagate_Compute.Main, 27.64).
2006-10-31 11:05:08.022360 26 UserTrace BIP2632I: Message received and propagated to 'out' terminal of MQ input node 'ConfigurationMessageFlow.InputNode'.
2006-10-31 11:05:08.022578 26 UserTrace BIP6060I: Parser type 'Properties' created on behalf of node 'ConfigurationMessageFlow.InputNode' to handle portion of incoming message of length 0 bytes beginning at offset '0'.
2006-10-31 11:05:08.022652 26 UserTrace BIP6061I: Parser type 'MQMD' created on behalf of node 'ConfigurationMessageFlow.InputNode' to handle portion of incoming message of length '364' bytes beginning at offset '0'. Parser type selected based on value 'MQHMD' from previous parser.
2006-10-31 11:05:08.023352 26 UserTrace BIP6061I: Parser type 'XML' created on behalf of node 'ConfigurationMessageFlow.InputNode' to handle portion of incoming message of length '203' bytes beginning at offset '364'. Parser type selected based on value 'XML' from previous parser.
2006-10-31 11:05:08.031901 26 UserTrace BIP4040I: The Execution Group 'EXEC_XNIBR002_BRK_BC_TESTFTP' has processed a configuration message successfully.
A configuration message has been processed successfully. Any configuration changes have been made and stored persistently.
No user action required.
2006-10-31 11:05:08.032645 26 UserTrace BIP2638I: The MQ output node 'ConfigurationMessageFlow.outputNode' attempted to write a message to queue 'SYSTEM.BROKER.EXECUTIONGROUP.REPLY' connected to queue manager 'XNIBR000'. The MQCC was '0' and the MQRC was '0'.
2006-10-31 11:05:08.032685 26 UserTrace BIP2622I: Message successfully output by output node 'ConfigurationMessageFlow.outputNode' to queue 'SYSTEM.BROKER.EXECUTIONGROUP.REPLY' on queue manager 'XNIBR000'.
2006-10-31 11:05:08.895449 26 UserTrace BIP2632I: Message received and propagated to 'out' terminal of MQ input node 'ConfigurationMessageFlow.InputNode'.
2006-10-31 11:05:08.895736 26 UserTrace BIP6060I: Parser type 'Properties' created on behalf of node 'ConfigurationMessageFlow.InputNode' to handle portion of incoming message of length 0 bytes beginning at offset '0'.
2006-10-31 11:05:08.895820 26 UserTrace BIP6061I: Parser type 'MQMD' created on behalf of node 'ConfigurationMessageFlow.InputNode' to handle portion of incoming message of length '364' bytes beginning at offset '0'. Parser type selected based on value 'MQHMD' from previous parser.
2006-10-31 11:05:08.896224 26 UserTrace BIP6061I: Parser type 'XML' created on behalf of node 'ConfigurationMessageFlow.InputNode' to handle portion of incoming message of length '193' bytes beginning at offset '364'. Parser type selected based on value 'XML' from previous parser.
2006-10-31 11:05:08.902388 26 UserTrace BIP7080I: Node 'PubSubControlMsgFlow.MatchingNode': The Publication Node with Subscription Point '' has received a message of type Publish.
The Publication Node with Subscription Point '' has started processing a message.
No user action required.
2006-10-31 11:05:08.903230 26 UserTrace BIP7081I: The Publication Node has matched 1 subscriptions to topic '$SYS/Broker/XNIBR000_BRK/Status/ExecutionGroup/EXEC_XNIBR002_BRK_BC_TESTFTP' for subscription point ''.
The Publication Node has matched subscriptions for the current publication and topic.
No user action required.
2006-10-31 11:05:08.903354 26 UserTrace BIP7082I: Node 'PubSubControlMsgFlow.MatchingNode': Publishing to destination XNICM000:SYSTEM.BROKER.ADMIN.REPLY:4366674d677253756233 for user TUXNCM.
A publication destination is being added to the list of destinations to XNICM000:SYSTEM.BROKER.ADMIN.REPLY:4366674d677253756233 for user TUXNCM.
No user action required.
2006-10-31 11:05:08.903450 26 UserTrace BIP7085I: Node 'PubSubControlMsgFlow.MatchingNode': The Publication Node has propagated a message to its output terminal for subscription point ''.
The Publication Node has propagated the current message to its output terminal.
No user action required.
2006-10-31 11:05:08.907145 26 UserTrace BIP2638I: The MQ output node 'PubSubControlMsgFlow.ListOutputNode' attempted to write a message to queue 'SYSTEM.BROKER.ADMIN.REPLY' connected to queue manager 'XNICM000'. The MQCC was '0' and the MQRC was '0'.
2006-10-31 11:05:08.907210 26 UserTrace BIP2622I: Message successfully output by output node 'PubSubControlMsgFlow.ListOutputNode' to queue 'SYSTEM.BROKER.ADMIN.REPLY' on queue manager 'XNICM000'.
2006-10-31 11:05:08.907585 26 UserTrace BIP4040I: The Execution Group 'EXEC_XNIBR002_BRK_BC_TESTFTP' has processed a configuration message successfully.
A configuration message has been processed successfully. Any configuration changes have been made and stored persistently.
No user action required.
2006-10-31 11:05:16.550882 24 UserTrace BIP2539I: Node 'BC_RIB_BatchGattung.SplitAndPropagate': Finished evaluating expression 'POSITION(blbLF IN InputRoot.BLOB.BLOB)' at (.BC_RIB_BatchGattung_SplitAndPropagate_Compute.Main, 27.21). This resolved to 'POSITION(X'0a' IN X'30303030303041415230303144453030304141523030313720202020332c3235252041415245414c2042414e4b2041472020202020202020202020202020454f2d4d544e2d4850462e532e3320462e312e322e323031312831312920312e322e45555245555232323730323031313032303133354445202a32303036303230313031322020202020202020202020202020202020202020203033302020202030303030302030303030303030504435353620322030302e30303030303030303032414143304330303034203038303431313041617265616c2042616e6b20414720202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020444520203132202020202020202020202020313230303630323031323030373032303146454220203030303030313030302e30303030303030303030303032343533303020455552313031323030363031323441415245414c2042414e4b204d544e2e4850462e532e3320203120323031313031333120333636203030302e3030303030303020203030303030302e303030304a303020202020202020202030303031303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030200a30303030303041424e30414e4e4c303030303230383439345658595341424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e44204e494b4b32323520313033393020202020202020202045555253544b39303936202020202020202038384e4c202a323030343032303230313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343031333041424e20414d524f20424b204d494e494c204e3232352020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3041344e4c303030303230383639325659425441424e20414d524f2042414e4b204e2e562e20202020202020202020202020202020424f4e55535a5430342f32362e30322e31302048412e53454e4743482e202020202045555253544b35393639323031303032323638384e4c202a323030343032323530313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a4331333320312030302e30303030303030303020424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032303941424e20414d524f20424b20424f2e5a5431302048534520203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030430a30303030303041424e3042534e4c303030303435353334332020202041424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e454e4420434543452020202020202020202020202020202045555253544b35393630202020202020202038384e4c202a323030343033303230313130313020202030343330383020202020202020202030333020202020303036393220303036393030304a4341443520312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032323541424e20414d524f20424b204f2e452e2051434320202020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030200a30303030303041424e30425a4e4c303030303231303839365659414641424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e4420414c552e46555420313433362020202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c2041324c462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3042314e4c303030303231303837305659414841424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e4420434f50502e465554203130363920202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c20433250462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3042324e4c303030303231303836325659414941424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e4953204f2e454e4420434f434f41465554203131363820202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e495320433243462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3042354e4c303030303231303833395659414c41424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e4420434f46462e46555420302c393537342020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c20433246462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3042374e4c303030303231303831335659414e41424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e44205355474152313120302c30363131202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c20533131462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3043424e4c303030303231303737315659415341424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e44204252454e5443522e2033342c3534202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c204f4649202020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3043434e4c303030303231303931325659415441424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e20454e44205748454154204655545552452020202020202045555253544b35393339202020202020202038384e4c202a32303034303231383031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313641424e20414d524f20424b205a54204f2e454e442057482e463320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030530a30303030303041424e3043444e4c303030303231303932305659415541424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e20454e4420434f464645452046555455524520202020202045555253544b35393339202020202020202038384e4c202a32303034303231383031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313641424e20414d524f20424b205a54204f2e454e4420434f2e463320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030530a30303030303041424e3043454e4c303030303231303933385659415641424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e20454e44205355474152204655545552452020202020202045555253544b35393339202020202020202038384e4c202a32303034303231383031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313641424e20414d524f20424b205a54204f2e454e442053552e463320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030530a30303030303041424e3043464e4c303030303231303934365659415741424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e20454e4420434f434f41204655545552452020202020202045555253544b35393339202020202020202038384e4c202a32303034303231383031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313641424e20414d524f20424b205a54204f2e454e4420434f2e463320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030530a30303030303041424e30434d4e4c303030303231303736335659424341424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a5430342f4f50454e20454e442057454226544543484e2e4241534b45542020202045555253544b35393339202020202020202038384e4c202a32303034303232373031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e3030303030303030304842425832423030303'. The result was '3990'. |
|
| Back to top |
|
|
| fschwarz |
| Posted: Mon Nov 13, 2006 9:24 am Post subject: Solution found |
|
|
Newbie
Joined: 31 Oct 2006
Posts: 7
|
Hi Jeff, all,
we finally found the solution for the performance issue described in this topic:
The traces where running !!!
If you investigate the trace statements listed above you will find lines like the following:
Finished evaluating expression 'POSITION(blbLF IN InputRoot.BLOB.BLOB)' at (.BC_RIB_BatchGattung_SplitAndPropagate_Compute.Main, 27.21). This resolved to 'POSITION(X'0a' IN X'30303030303041415230303144453030304141523030313720202020332c3235252041415245414c2042414e4b2041472020202020202020202020202020454f2d4d544e2d4850462e532e3320462e312e322e323031312831312920312e322e45555245555232323730323031313032303133354445202a32303036303230313031322020202020202020202020202020202020202020203033302020202030303030302030303030303030504435353620322030302e30303030303030303032414143304330303034203038303431313041617265616c2042616e6b20414720202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020444520203132202020202020202020202020313230303630323031323030373032303146454220203030303030313030302e30303030303030303030303032343533303020455552313031323030363031323441415245414c2042414e4b204d544e2e4850462e532e3320203120323031313031333120333636203030302e3030303030303020203030303030302e303030304a303020202020202020202030303031303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030200a30303030303041424e30414e4e4c303030303230383439345658595341424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e44204e494b4b32323520313033393020202020202020202045555253544b39303936202020202020202038384e4c202a323030343032303230313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343031333041424e20414d524f20424b204d494e494c204e3232352020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3041344e4c303030303230383639325659425441424e20414d524f2042414e4b204e2e562e20202020202020202020202020202020424f4e55535a5430342f32362e30322e31302048412e53454e4743482e202020202045555253544b35393639323031303032323638384e4c202a323030343032323530313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a4331333320312030302e30303030303030303020424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032303941424e20414d524f20424b20424f2e5a5431302048534520203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030430a30303030303041424e3042534e4c303030303435353334332020202041424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e454e4420434543452020202020202020202020202020202045555253544b35393630202020202020202038384e4c202a323030343033303230313130313020202030343330383020202020202020202030333020202020303036393220303036393030304a4341443520312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032323541424e20414d524f20424b204f2e452e2051434320202020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030200a30303030303041424e30425a4e4c303030303231303839365659414641424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e4420414c552e46555420313433362020202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c2041324c462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3042314e4c303030303231303837305659414841424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e4420434f50502e465554203130363920202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c20433250462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3042324e4c303030303231303836325659414941424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e4953204f2e454e4420434f434f41465554203131363820202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e495320433243462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3042354e4c303030303231303833395659414c41424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e4420434f46462e46555420302c393537342020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c20433246462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3042374e4c303030303231303831335659414e41424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e44205355474152313120302c30363131202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c20533131462020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3043424e4c303030303231303737315659415341424e20414d524f2042414e4b204e2e562e202020202020202020202020202020204d494e494c204f2e454e44204252454e5443522e2033342c3534202020202020202045555253544b39303636202020202020202038384e4c202a323030343032313630313120202020202020202020202020202020202020202030333020202020303030303020303030303030304a5731333320312030302e30303030303030303048424258324530303031203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203234202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313341424e20414d524f20424b204d494e494c204f4649202020203320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030330a30303030303041424e3043434e4c303030303231303931325659415441424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e20454e44205748454154204655545552452020202020202045555253544b35393339202020202020202038384e4c202a32303034303231383031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313641424e20414d524f20424b205a54204f2e454e442057482e463320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030530a30303030303041424e3043444e4c303030303231303932305659415541424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e20454e4420434f464645452046555455524520202020202045555253544b35393339202020202020202038384e4c202a32303034303231383031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313641424e20414d524f20424b205a54204f2e454e4420434f2e463320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030530a30303030303041424e3043454e4c303030303231303933385659415641424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e20454e44205355474152204655545552452020202020202045555253544b35393339202020202020202038384e4c202a32303034303231383031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313641424e20414d524f20424b205a54204f2e454e442053552e463320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030530a30303030303041424e3043464e4c303030303231303934365659415741424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a542030342f4f50454e20454e4420434f434f41204655545552452020202020202045555253544b35393339202020202020202038384e4c202a32303034303231383031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e30303030303030303048424258324230303032203034303533323241424e20414d524f2042616e6b204e2e562e2020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020204e4c20203231202020202020202020202020202020202020202020202020202020202020202020203030303030303030312e30303030303030303030303038312020202020455552313030323030343032313641424e20414d524f20424b205a54204f2e454e4420434f2e463320202020202020202020303030203030302e3030303030303020203030303030302e3030303020303020202020202020202030303030303031203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030203030303030303020303030303030302030303030303030530a30303030303041424e30434d4e4c303030303231303736335659424341424e20414d524f2042414e4b204e2e562e202020202020202020202020202020205a5430342f4f50454e20454e442057454226544543484e2e4241534b45542020202045555253544b35393339202020202020202038384e4c202a32303034303232373031312020202020202020202020202020202020202020203033302020202030303030302030303030303030494431333320312030302e3030303030303030304842425832423030303'. The result was '3990'.
Please note the trailing string which is the current message (28MBs). this message is transformed from BLOB to string explaining the CPU utilization and memcopies we found using truss.
You will also notice that this is a cutted string (not the full 28MB) but it looks like the whole BLOB is translated to string and afterwards cutted to the maximum display length. That was around 10secs for converting the string on our environment which finally led to a throughput of 1 message in 20 secs (Position and Substring behave the same an log the BLOB) instead of normal throughput (~around 730 msgs/sec).
Switching off the trace resolved the whole issue (no log written, no BLOB -> String conversion).
Thanks @ all for you help in this issue. |
|
| Back to top |
|
|
|
|
