ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Log4j vs. Log queue vs. Trace node.

Post new topic  Reply to topic Goto page 1, 2  Next
 Log4j vs. Log queue vs. Trace node. « View previous topic :: View next topic » 
Author Message
brokerguy
PostPosted: Mon Mar 18, 2013 1:46 pm    Post subject: Log4j vs. Log queue vs. Trace node. Reply with quote

Apprentice

Joined: 18 Mar 2013
Posts: 26
Location: Cosmos

Hi All,

I had earlier created a re-usable utility which involved creating a wrapper around log4j which was invoked from ESQL using extern function. It worked OK in production and provided the ability to set the appropriate log levels.
Now, I was seeing the service proxy pattern and it appears like it uses a Log queue.

I was wondering which one according to you is a good design..having synchronous log writes to file using log4j or use asynch log queue or have Trace nodes ?
Trace node would not allow inline logging from a ESQL or Java compute, however, would like to know your views.

Thanks.
Back to top
View user's profile Send private message
goffinf
PostPosted: Tue Mar 19, 2013 1:28 am    Post subject: Reply with quote

Chevalier

Joined: 05 Nov 2005
Posts: 401

I assume your Log4j 'wrapper' is configured to use an appender which ASYNCHronously writes to your log file, so it isn't blocking the Message Broker thread any longer than the hand-off to Log4j takes ?
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Mar 19, 2013 4:06 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Don't use log4j.

Use java.util.logging.

There's several different kinds of logging.

What the appropriate design is depends on which type of logging you're trying to do.

But regardless, all kinds of logging from Java should NOT use log4j and SHOULD use java.util.logging.

It's included in the base SDK!
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Tue Mar 19, 2013 4:07 am    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

mqjeff advocates using http://docs.oracle.com/javase/6/docs/api/java/util/logging/package-summary.html and I agree with his recommendation.

The situation is not either/or or mutually exclusive.

Trace nodes are useful in test environments to help understand behaviors.

Logger is useful in production to record a summary of events.

We turn Trace nodes off in production because we do not want to write sensitive information to disk.

Using a queue to buffer log writes may be a better idea in high-volume applications.

DailyRollingFileAppender is awesome.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
smdavies99
PostPosted: Tue Mar 19, 2013 4:27 am    Post subject: Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.

lancelotlinc wrote:


Using a queue to buffer log writes may be a better idea in high-volume applications.


I'll second this approach. It is far more flexible and handles peak loads a lot better.
Just be careful to think about the transactionality of the MQOutput Nodes you use to send data to the loggers. Do you really want the input message log event to be rolled back in the event of an error? better to make that MQPUT/MQCMIT immediate rather than wait for the whole flow processing to complete and be committed under transaction control.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
Esa
PostPosted: Tue Mar 19, 2013 4:50 am    Post subject: Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

smdavies99 wrote:
lancelotlinc wrote:


Using a queue to buffer log writes may be a better idea in high-volume applications.


I'll second this approach. It is far more flexible and handles peak loads a lot better.


But if the application that processes the log queue is a Message Broker flow, you will eventually have to use java.util.logging or equivalent to write the messages on a file.

I think it's quite usual to use a log file as a last resort, for logging messages like "could not put the log message in the log queue". But a better alternative would be to write in the system log with MbService class.

The OP is obviously using logging for tracing his java code, in which case writing to system log is not the best option.

It would be nice if you could write to user trace from java. Maybe MbService class could be extended with methods that allow writing to user trace?

But perhaps the reason for not being able to write custom user trace messages is that IBM support does not want users to fill the traces with rubbish. The messages would end up in service trace too and make it more difficut for the support team to find the relevant stuff?
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Tue Mar 19, 2013 5:05 am    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

Esa wrote:
smdavies99 wrote:
lancelotlinc wrote:


Using a queue to buffer log writes may be a better idea in high-volume applications.


I'll second this approach. It is far more flexible and handles peak loads a lot better.


a better alternative would be to write in the system log with MbService class.


Only true if your log message is within the length constraints. Often, my messages would not fit and be truncated by the syslog. Especially if writing a stack trace.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Tue Mar 19, 2013 5:21 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Esa wrote:
It would be nice if you could write to user trace from java. Maybe MbService class could be extended with methods that allow writing to user trace?

The .NET interface allows this.

Esa wrote:
But perhaps the reason for not being able to write custom user trace messages is that IBM support does not want users to fill the traces with rubbish. The messages would end up in service trace too and make it more difficut for the support team to find the relevant stuff?

It's safer to assume that the Java API was relatively stabilized before the requirement to log to user trace was properly understood.
Back to top
View user's profile Send private message
brokerguy
PostPosted: Tue Mar 19, 2013 7:47 am    Post subject: Reply with quote

Apprentice

Joined: 18 Mar 2013
Posts: 26
Location: Cosmos

smdavies99 wrote:
lancelotlinc wrote:


Using a queue to buffer log writes may be a better idea in high-volume applications.


I'll second this approach. It is far more flexible and handles peak loads a lot better.
Just be careful to think about the transactionality of the MQOutput Nodes you use to send data to the loggers. Do you really want the input message log event to be rolled back in the event of an error? better to make that MQPUT/MQCMIT immediate rather than wait for the whole flow processing to complete and be committed under transaction control.


Agree. However, do you agree that the log messages need to have the right system time recorded and held in memory before writing to a queue. The intent is not only to log incoming/outgoing messages but also to perform in-line logging from ESQL. Using a queue to store the messages before writing to a physical application log file will introduce some obvious delays vs. writing to the log file directly.

What do you think ?
Back to top
View user's profile Send private message
brokerguy
PostPosted: Tue Mar 19, 2013 7:52 am    Post subject: Reply with quote

Apprentice

Joined: 18 Mar 2013
Posts: 26
Location: Cosmos

Esa wrote:
But a better alternative would be to write in the system log with MbService class.


The logging here refers to application logging. Logs that are generated when a service (soap, mq, etc) is invoked, when the exception path writes errors.

Is it appropriate to perform application logging in to system log ? Your inputs will be much appreciated.
Back to top
View user's profile Send private message
Esa
PostPosted: Tue Mar 19, 2013 7:59 am    Post subject: Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

brokerguy wrote:
Esa wrote:
But a better alternative would be to write in the system log with MbService class.


The logging here refers to application logging. Logs that are generated when a service (soap, mq, etc) is invoked, when the exception path writes errors.

Is it appropriate to perform application logging in to system log ? Your inputs will be much appreciated.


It's not.
Back to top
View user's profile Send private message
brokerguy
PostPosted: Tue Mar 19, 2013 1:24 pm    Post subject: Reply with quote

Apprentice

Joined: 18 Mar 2013
Posts: 26
Location: Cosmos

mqjeff wrote:
Don't use log4j.

Use java.util.logging.

There's several different kinds of logging.

What the appropriate design is depends on which type of logging you're trying to do.

But regardless, all kinds of logging from Java should NOT use log4j and SHOULD use java.util.logging.

It's included in the base SDK!


would like to understand the rationale of not recommending log4j. Not sure java.util.logging provides appenders. Also, my intention is to have common log file for a single broker (all EGs in the broker writing to the same log file). Would like to know your views on this.
Back to top
View user's profile Send private message
smdavies99
PostPosted: Tue Mar 19, 2013 8:04 pm    Post subject: Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.

brokerguy wrote:
Also, my intention is to have common log file for a single broker (all EGs in the broker writing to the same log file). Would like to know your views on this.


It may get very large
It may have bits of logging for one flow all mixed up with bits of logging for other flow(s)

IF you log to a DB you can do all sorts of matching between input and output messages.
The log table structure I use has TWO timestamp fields. the first is the time of the log event. The second is the time the log record was written.
You can also put a DB job in place to purge the log tables of data older than say 7 days.

At my current site we log 250,000 records a day. This is just input messages. Think how you are going to handle a file with that many records?
Think how you are going to search for problems, repeating issues etc?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Wed Mar 20, 2013 3:58 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

brokerguy wrote:
would like to understand the rationale of not recommending log4j. Not sure java.util.logging provides appenders.

mqjeff wrote:
It's included in the base SDK!


It's used by many IBM products, including WAS.

It's faster. It's *much* easier to work with, since it's NOT an additional set of jar files you have to deploy and then worry about which classloader has loaded them and which classloader needs to load the properties.

And, again...

mqjeff wrote:
It's included in the base SDK!


Using log4j is like tying a large rock to a chain, and using that instead of the brakes that come on your car.

brokerguy wrote:
Also, my intention is to have common log file for a single broker (all EGs in the broker writing to the same log file). Would like to know your views on this.


It's fine as long as you take the time to make sure you can change this later without having to rewrite any significant code.
Back to top
View user's profile Send private message
brokerguy
PostPosted: Wed Mar 20, 2013 7:20 am    Post subject: Reply with quote

Apprentice

Joined: 18 Mar 2013
Posts: 26
Location: Cosmos

mqjeff wrote:


It's used by many IBM products, including WAS.

It's faster. It's *much* easier to work with, since it's NOT an additional set of jar files you have to deploy and then worry about which classloader has loaded them and which classloader needs to load the properties.



Quick query- While using java logging, did you write custom code for handling file rollovers ? This comes out of the box with log4j. right ?
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Log4j vs. Log queue vs. Trace node.
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.