| Author |
Message
|
| sleepyjamie |
 Posted: Tue Aug 25, 2015 6:57 am Post subject: Logging XML instead of wireframe format |
 |
|
Centurion
Joined: 29 Apr 2015
Posts: 135
|
Currently a lot of our User Trace nodes are logging as follows.
${Body}
This outputs the wireframe format. We would like this to output as human readable XML format instead. For example, when I have a SOAP Request I would like the user trace node to log the request/response as XML.
Is there a way to tell the trace nodes to output as XML instead of wireframe format? |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Aug 25, 2015 7:06 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
This isn't really the job of the Trace node.
You can put ESQL expressions in ${}. But you're better off using the log functions of the transformation interface you're using.
ESQL provides the LOG statement which can output to usertrace, and the .NET api also provides methods for logging to usertrace.
The Java API provides methods on the MBService object to log to the system log (BIPmessages).
Off the top of my head, I don't remember what the Map provides.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 25, 2015 7:08 am Post subject: Re: Logging XML instead of wireframe format |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sleepyjamie wrote: |
| Is there a way to tell the trace nodes to output as XML instead of wireframe format? |
No, because the Trace node is concerned with the message tree that's being processed, which may or may not have any connection to an actual input message as it moves through the flow.
| sleepyjamie wrote: |
| For example, when I have a SOAP Request I would like the user trace node to log the request/response as XML. |
If you want to log the incoming XML, log the incoming XML. Don't use Trace nodes for audit logging. Or anything other than tracing.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sleepyjamie |
| Posted: Tue Aug 25, 2015 7:19 am Post subject: Re: Logging XML instead of wireframe format |
|
|
Centurion
Joined: 29 Apr 2015
Posts: 135
|
We have IBM contractors who developed code for us, so we are just following their implementation. The trace logs are typically enabled when something goes wrong in production, and are only enabled for a short time period to capture data to analyze issues related 3rd party Web Services.
The trace nodes output the SOAP request/response to a Trace log file. We want to have external logging products such as splunk/logstash that consume these trace logs which allow them to be analyzed by operations in a much more human readable format.
Using the wireframe format is not human friendly for operations folks, nor for the developers.
Changing all the code at this point would be cumbersome and likely too risky. I'm wondering if it is possible within the Trace node to tell it to serialize the tree hierarchy as XML rather than wireframe format.
psuedo code...
TO_XML(${InputRoot.XMLNSC}) |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Aug 25, 2015 7:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You could ASBITSTREAM. But it would have to be inside the ${}. And I don't know if it works from a trace node.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| sleepyjamie |
| Posted: Tue Aug 25, 2015 7:22 am Post subject: |
|
|
Centurion
Joined: 29 Apr 2015
Posts: 135
|
| mqjeff wrote: |
| You could ASBITSTREAM. But it would have to be inside the ${}. And I don't know if it works from a trace node. |
Yes! I will try that out! |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 25, 2015 7:27 am Post subject: Re: Logging XML instead of wireframe format |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sleepyjamie wrote: |
| We have IBM contractors who developed code for us, so we are just following their implementation. The trace logs are typically enabled when something goes wrong in production, and are only enabled for a short time period to capture data to analyze issues related 3rd party Web Services. |
Which is a valid use case for a Trace node, and one of the reasons they can be enabled and disabled by administrative means (rather than a code deployment).
| sleepyjamie wrote: |
| The trace nodes output the SOAP request/response to a Trace log file. We want to have external logging products such as splunk/logstash that consume these trace logs which allow them to be analyzed by operations in a much more human readable format. |
The Trace nodes are not intended to be used by operators, or as any kind of log. They're intended to be used by developers to diagnose code problems.
| sleepyjamie wrote: |
| Using the wireframe format is not human friendly for operations folks, nor for the developers. |
The developers will gain much benefit from being able to easily use the output of a Trace node. Especially during development. The operations folk should never need to look at it.
| sleepyjamie wrote: |
| Changing all the code at this point would be cumbersome and likely too risky. |
Fair point. But if all you want is the XML, why not simply obtain that with the inbuilt facilities? It might not be as "splunky" as you're looking for, but this is IIB not WAS and the rules are different in here.
| sleepyjamie wrote: |
| I'm wondering if it is possible within the Trace node to tell it to serialize the tree hierarchy as XML rather than wireframe format. |
No means that I'm aware of.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 25, 2015 7:28 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sleepyjamie wrote: |
| Yes! I will try that out! |
Do please post the answer. I'm always keen to have my mind expanded. I was firmly of the opinion the Trace node did pattern matching & didn't support ESQL.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Aug 25, 2015 7:29 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You could also write a logstash filter that would transform the logical message tree output by the Trace node into a real XML message, or at least a bitstream of it.
It might not be terribly easy, but I'm reasonably sure the capability is there.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 25, 2015 7:33 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| It might not be terribly easy, but I'm reasonably sure the capability is there. |
I'm pleased to see your mastery of understatement remains undiminished 
But I agree - I'm sure you can write any code in any language to perform any task. Eventually.
I'll settle for being really pleased when the Trace node executes an ASBITSTREAM. Make my infrastructure a lot easier.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Aug 25, 2015 7:35 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Vitor wrote: |
| mqjeff wrote: |
| It might not be terribly easy, but I'm reasonably sure the capability is there. |
I'm pleased to see your mastery of understatement remains undiminished
But I agree - I'm sure you can write any code in any language to perform any task. Eventually.
I'll settle for being really pleased when the Trace node executes an ASBITSTREAM. Make my infrastructure a lot easier. |
Mad skillz.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| sleepyjamie |
| Posted: Tue Aug 25, 2015 7:35 am Post subject: |
|
|
Centurion
Joined: 29 Apr 2015
Posts: 135
|
| Vitor wrote: |
| sleepyjamie wrote: |
| Yes! I will try that out! |
Do please post the answer. I'm always keen to have my mind expanded. |
Absolutely!
I can agree that trace nodes *should* only be for developers, but I work in a large organization of 12,000+ where developers are usually brought in when system instability is of concern.
In this case, we are upgrading the message flows to use new version of 3rd party Web Services. As such the DevOps team will carry out tests in staging and enable trace logging to audit the system behaviour, before pushing to production.
What I am trying to do is capture RAW XML so that if a WebService request fails, the DevOps person can use this XML and test the Web Service independently using SOAP UI for the specific SOAP request.
While this might not be the best approach, this is what IBM developed for us and unfortunately we are stuck with it. I'm simply trying to ease the pain for the DevOps team. |
|
| Back to top |
|
|
| sleepyjamie |
| Posted: Tue Aug 25, 2015 7:36 am Post subject: |
|
|
Centurion
Joined: 29 Apr 2015
Posts: 135
|
| mqjeff wrote: |
You could also write a logstash filter that would transform the logical message tree output by the Trace node into a real XML message, or at least a bitstream of it.
It might not be terribly easy, but I'm reasonably sure the capability is there. |
Yeah I considered that but I think it would be very painful to implement.
That's why I'm hoping IIB can output XML since XML codecs are natively supported in splunk/logstash.
I will keep trying this out and report back with my findings. Thanks! |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Aug 25, 2015 7:37 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 25, 2015 7:39 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Well if it acts on the message tree as well as the static functions, you're sorted.
This is why my most worthy associate is most worthy. In all his incarnations.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
