| Author |
Message
|
| jolloqui |
 Posted: Thu Mar 15, 2007 7:47 am Post subject: Flow works in Debug - fails out of debug |
 |
|
Novice
Joined: 15 Mar 2007
Posts: 20
|
Hello,
I've spent days trying to troubleshoot this  and I've given up - need help 
I'm using WBI v5 - in a Win2K server. The section of the flow where the failure seems to occur is a Compute node + XMLTransformation node + ResetContent (from Blob into XML).
If I 'short-circuit' this part, it works like a charm. If I re-instate them in the flow AND am in debug, still works like a charm. If I detach from debug... it fails.
Any ideas?
Thanks!
Pepe |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Mar 15, 2007 7:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
What does it fail with? Any error messages / codes / log entries?
What other troubleshooting efforts have you made?
(There's no point us suggesting things you've already tried and/or eliminated...  )
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jolloqui |
| Posted: Thu Mar 15, 2007 8:05 am Post subject: |
|
|
Novice
Joined: 15 Mar 2007
Posts: 20
|
The flow has about 20 nodes, so I started "short-circuting" those that could and have identified the section that fails.
I've added trace nodes and nothing seems to go bad (that I can see). Also added the 'On' for the XMLTransformation trace.
From a queue standpoint - the message flows to the 'dead.letter' system queue.
I need to keep the original XML. So we copy the XML into the 'MQRFH2' section in order to do the transformation. Then the original XML is retrieved for latter use.
| Code: |
| SET OutputRoot.MQRFH2.usr.Original_XML = InputRoot.XML; |
I have a feeling it has to do with the size of the XML - if it has 7K it fails, if it has 5K sometimes it works.
The weird part is that it always works in debug mode - with or without breakpoints. The same XML will fail if out of debug (e.g. ends in the 'dead.letter' queue)
Thanks!! |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Mar 15, 2007 8:27 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jolloqui wrote: |
From a queue standpoint - the message flows to the 'dead.letter' system queue. |
What's the reason code in the dead letter header?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jolloqui |
| Posted: Thu Mar 15, 2007 8:36 am Post subject: |
|
|
Novice
Joined: 15 Mar 2007
Posts: 20
|
here is the header:
| Code: |
DLH ERROR.IN GEMI.OMNI.PROD001 "ä WebSphereMQIntegrator5 2007031514324500
|
after this, a copy of the XML is included...
hope this helps...  |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Mar 15, 2007 8:42 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Not especially, the reason code is a decimal value & hence shows as spaces in your post.....
Can you find and post the reason code? You'll find the layout in the Application Programming Refernce. Or utilities like Explorer or RFHUTIL will decode it for you.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jolloqui |
| Posted: Thu Mar 15, 2007 8:49 am Post subject: |
|
|
Novice
Joined: 15 Mar 2007
Posts: 20
|
I copied from rfhutil using the 'XML' view.
If i use the 'both' (character and hex, I suppose) this is what I get:
| Code: |
00000000 DLH .... ....ERRO 444C4820 01000000 00000100 4552524F
00000016 R.IN 522E494E 20202020 20202020 20202020
00000032 20202020 20202020 20202020 20202020
00000048 GEMI 20202020 20202020 20202020 47454D49
00000064 .OMNI.PR OD001 2E4F4D4E 492E5052 4F443030 31202020
00000080 20202020 20202020 20202020 20202020
00000096 "... 20202020 20202020 20202020 22020000
00000112 .... .... E4040000 20202020 20202020 0B000000
00000128 WebSpher eMQInteg 57656253 70686572 654D5149 6E746567
00000144 rator5 2007 7261746F 72352020 20202020 32303037
00000160 03151537 0400<?xm 30333135 31353337 30343030 3C3F786D
|
would the E404 (bold) be the error code?
("00000112 .... .... E4040000 20202020 20202020 0B000000") |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Mar 15, 2007 9:13 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jolloqui wrote: |
I copied from rfhutil using the 'XML' view.
....
would the E404 (bold) be the error code?
("00000112 .... .... E4040000 20202020 20202020 0B000000") |
I hope not - the reason code should be a lot further up the structure!
The code quoted also doesn't make sense at first glance. Can you double check it - you'll find RFHUTIL has a DLQ tab which will extract the reason code automatically.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| EddieA |
| Posted: Thu Mar 15, 2007 10:32 am Post subject: |
|
|
Jedi
Joined: 28 Jun 2001
Posts: 2453
Location: Los Angeles
|
If the Broker has put the message in the DLQ, then it'll have the "standard" code, which doesn't tell you anything.
You need to look in the Event Viewer to see why.
Cheers.
_________________
Eddie Atherton
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0 |
|
| Back to top |
|
|
| jolloqui |
| Posted: Fri Mar 16, 2007 1:09 am Post subject: |
|
|
Novice
Joined: 15 Mar 2007
Posts: 20
|
Here is a dump of two 'Event Viewer' logs that start just after the request failed - if in Debug mode there is no event logged at all.
Event 1--
| Code: |
( BR_PROD001.DataLoads-005 ) Broker process terminating abnormally: The following diagnostic information will be required when contacting IBM: '
Severe Abend Error detected.
For full details see Abend File: C:/Program Files/IBM/WebSphere Business Integration Message Brokers/errors/BR_PROD001.DataLoads-005.5416.4500.Abend
A summary of the Error follows:
An Unhandled Exception detected in process 5416, on thread 0x1194.
Type: EXCEPTION_ACCESS_VIOLATION (C0000005).
Address: 001B:77FCB940.
The thread could not write to memory address 0x00000000.
'.
A broker process is terminating abnormally.
Contact your IBM support center.
|
Event 2 -- (right after Event1)
| Code: |
( BR_PROD001.DataLoads-005 ) Message broker internal error: diagnostic information 'As a result of a Severe Abend Error, a MiniDump has been written to: C:/Program Files/IBM/WebSphere Business Integration Message Brokers/errors/BR_PROD001.DataLoads-005.5416.4500.dmp'.
An internal software error has occurred in the message broker. Further messages will indicate the effect of this error on the broker's transactions. The diagnostic information associated with this message is: 'As a result of a Severe Abend Error, a MiniDump has been written to: C:/Program Files/IBM/WebSphere Business Integration Message Brokers/errors/BR_PROD001.DataLoads-005.5416.4500.dmp'.
Shutdown and restart the message broker. If the problem continues to occur, then restart the system. If the problem still continues to occur contact your IBM support center.
|
Here is a partial of the 'abend' file mentioned in Event1 - the full thing is huge.
| Code: |
Abend Record for PID: 5416 TID: 0x1194 on: Fri Mar 16 08:38:54 2007
---> Location <---
File: F:\build\S500_P\src\CommonServices\Win32\ImbAbend.cpp
Line: 1386
Function: ImbAbend::unhandledExceptionFilterInternal()
AbendAction: 3
---> Inserts <---
An Unhandled Exception detected in process 5416, on thread 0x1194.
Type: EXCEPTION_ACCESS_VIOLATION (C0000005).
Address: 001B:77FCB940.
The thread could not write to memory address 0x00000000.
---> Last Errors <---
Note these error codes are not necessarily reliable
GetLastError() returned: 0
errno returned: 0
---> Sys Info <---
Machine Name: 49415-OSAPP1
Process User Name: Bradmin
No of Processors: 4
Processor Type: x86 Family 15 Model 2 Stepping 9
OS: Microsoft Windows 2000 Advanced Server Service Pack 4 (Build 2195)
Current Type: Multiprocessor Free
Command Line: "C:/Program Files/IBM/WebSphere Business Integration Message Brokers/bin/DataFlowEngine.exe" "BR_PROD001" "3f1e01da-0c01-0000-0080-de01ef5b66ce" "DataLoads-005" "0"
CMVC: @(#) 1.18.1.1 CommonServices/Win32/ImbAbend.cpp, CommonServices, S500, S500-CSD02 03/10/16 20:33:50 [12/12/03 16:06:51]
Lib Version: 4.0.5.0
---> Process Status <---
Memory:
Usage Load: 67%
Total Physical Memory: 2147483647 bytes
Available Physical Memory: 1307459584 bytes
Total PageFile: 4294967295 bytes
Available PageFile: 4294967295 bytes
Total Virtual Memory: 2147352576 bytes
Available Virtual Memory: 1709916160 bytes
Working Set:
Min: 204800 bytes
Max: 1413120 bytes
Timings:
Creation Time: D-16 M-3 Y-2007 : 8:36:31.12
Kernel Time: 1484 ms
User Time: 16078 ms
---> Stack dump for the faulting thread (0x1194), with malloc disallowed. <---
Registers:
EAX=0x00000000 EBX=0x1074BE80 ECX=0x7FF9F000 EDX=0x01100608
ESI=0x00000000 EDI=0x01100000 EIP=0x77FCB940 ESP=0x1074BCE8
EBP=0x1074BD7C EFlags=0x00010217
CS=0x001B DS=0x0023 SS=0x0023 ES=0x0023 FS=0x003B GS=0x0000
Stack Backtrace:
# ChildEBP RetAddr Param#1 Param#2 Param#3 Param#4 Fn-Loc'n : Module!Function+Offset [File Name # Line+Offset @ Address]
00 1074BD7C 78001E00 (01100000 00000000 1074BE80 00002000) 77FCB940 : ntdll!RtlFreeHeap+0x276<NLN:487>
01 1074BDC4 0BC0E3FB (1074BE80 0BC09A9E 1074BE80 1074D074) 78001E00 : MSVCRT!free+0x50<NLN:487>
02 1074BDCC 0BC09A9E (1074BE80 1074D074 00000000 130B7680) 0BC0E3FB : CMNode50!??3@YAXPAX@Z+0x9 (FPO: [1,0,0])<NLN:487>
03 1074CE7C 0BC04531 (0BC12020 0BC12270 0000016B 0BC12024) 0BC09A9E : CMNode50!_TraceRpt+0x24e (FPO: [EBP: 0x1074D074] [9,1062,4])<NLN:487>
04 1074CED0 0BC05653 (1074D074 130B7680 1074CEFC 0F9F1258) 0BC04531 : CMNode50!?CMNodeXml@@YAJPAX0AAVCMBufP@@@Z+0x1f1 (FPO: [3,8,2])<NLN:487>
05 1074CF30 0BC05D65 (0F9F1E68 1074D05C 1074D004 1074D074) 0BC05653 : CMNode50!_CMNode_evaluateCPP+0x353 (FPO: [EBP: 0x1074D140] [4,16,4])<NLN:487>
06 1074CF68 2D963DB2 (0F9F1E68 1074D05C 1074D004 1074D074) 0BC05D65 : CMNode50!__CMNode_evaluate+0x45 (FPO: [4,0,0])<NLN:487>
07 1074D140 00ECAEA3 (1074D20C 0F9F1E30 0F948334 1074D20C) 2D963DB2 : imbdfplg!ImbCniNode::evaluate+0x1e2 [F:\build\S500_P\src\DataFlowEngine\PluginInterface\ImbCniNode.cpp # 1282+0 @ 0x2D963DB2]
08 1074D1B0 00ECAC72 (1074D20C 2E704F00 1074D7F0 0F9482D0) 00ECAEA3 : DataFlowDLL!ImbDataFlowTerminal::evaluate+0x1c3 (FPO: [EBP: 0x0F948334] [1,22,3]) [F:\build\S500_P\src\DataFlowEngine\ImbDataFlowTerminal.cpp # 285+0 @ 0xECAEA3]
09 1074D1E8 2D7ED734 (1074D20C 0F9482D0 0F9482D0 1074D63C) 00ECAC72 : DataFlowDLL!ImbDataFlowTerminal::propagate+0x142 (FPO: [EBP: 0x2E704F00] [1,8,4]) [F:\build\S500_P\src\DataFlowEngine\ImbDataFlowTerminal.cpp # 251+9 @ 0xECAC69]
0A 1074D288 2D7ED432 (1074D7D8 1074D5B0 0F948308 1074D7D8) 2D7ED734 : imbdfsql!ImbComputeNode::propagate+0x1d4 (FPO: [EBP: 0x0F9482D0] [2,34,4]) [F:\build\S500_P\src\DataFlowEngine\ImbComputeNode.cpp # 518+0 @ 0x2D7ED734]
0B 0F9482D0 0F948248 (00000000 00000000 00000000 00000000) 2D7ED432 : imbdfsql!ImbComputeNode::evaluate+0xa22 [F:\build\S500_P\src\DataFlowEngine\ImbComputeNode.cpp # 477+26 @ 0x2D7ED418]
0C 2D80685C 2D7EAF1F (2D803D7A 2D803E46 2D803D74 2D803D6E) 0F948248 : ***StackWalk Warning: Module does not appear to exist. Following frames may be wrong. Error Code: 0*** <Unknown>!<No Symbol Found>, Error Code: 126 <NLN:126>
0D 2D803D80 25FF2D80 (2D8060EC 60F025FF 25FF2D80 2D806104) 2D7EAF1F : imbdfsql!ImbComputeNode::type+0x12f<NLN:487>
0E 604025FF 00000000 (00000000 00000000 00000000 00000000) 25FF2D80 : ***StackWalk Warning: Module does not appear to exist. Following frames may be wrong. Error Code: 299*** <Unknown>!<No Symbol Found>, Error Code: 126 <NLN:126>
|
Sorry for such a long post - hope this helps
Thanks! |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Mar 16, 2007 1:20 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Look a lot like a null pointer from the broker. I'd seriously consider a PMR at this point.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Mar 16, 2007 3:50 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Abends should always result in opening a PMR.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| jolloqui |
| Posted: Mon Mar 19, 2007 1:55 am Post subject: |
|
|
Novice
Joined: 15 Mar 2007
Posts: 20
|
Will do that - Thanks for the help!
|
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Mar 19, 2007 3:01 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| jolloqui wrote: |
I need to keep the original XML. So we copy the XML into the 'MQRFH2' section in order to do the transformation. Then the original XML is retrieved for latter use.
| Code: |
| SET OutputRoot.MQRFH2.usr.Original_XML = InputRoot.XML; |
I have a feeling it has to do with the size of the XML - if it has 7K it fails, if it has 5K sometimes it works.
The weird part is that it always works in debug mode - with or without breakpoints. The same XML will fail if out of debug (e.g. ends in the 'dead.letter' queue)
Thanks!! |
Have you read up on the capabilities of the usr folder in the RFH header?
It is designed to host only a property = value flat structure.
Your value is an XML document with multiple levels. If you can transform that into a non xml flat structure before saving it you might have a better chance...
The additional question I have here would be a design question:
Why put the original XML into the RFH. Why not into the environment or on a special queue with a correl id and retrieve it by correl id?
If you are doing an aggregation you could just send the original message to the aggregation...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jolloqui |
| Posted: Mon Mar 19, 2007 3:21 am Post subject: |
|
|
Novice
Joined: 15 Mar 2007
Posts: 20
|
| fjb_saper wrote: |
Have you read up on the capabilities of the usr folder in the RFH header?
It is designed to host only a property = value flat structure.
Your value is an XML document with multiple levels. If you can transform that into a non xml flat structure before saving it you might have a better chance... |
We've tried saving the XML as a flat structure (both Blob and character) with no consistent results - it works Ok for smaller XMLs (<5Kb) but fails for the larger ones (>7Kbs).
| fjb_saper wrote: |
The additional question I have here would be a design question:
Why put the original XML into the RFH. Why not into the environment or on a special queue with a correl id and retrieve it by correl id? |
It seemed simpler that way - and it worked Ok (at least while debugging). We'll try using a queue as you mention - makes more sense.
Thanks!
(I'll be back with news - hopefully positive)
Jose |
|
| Back to top |
|
|
|
|
