| Author |
Message
|
| EricCox |
 Posted: Thu Mar 13, 2014 9:01 am Post subject: Lots about log4j - Existing:Not Working - And Future |
 |
|
Master
Joined: 08 Apr 2011
Posts: 292
|
To all,
I have a two part question. The first part is a series of questions about an existing service which uses log4j but it doesn't appear to be working correctly.
Q1.) I have evidence log4j is implemented in a JCN. I cannot find any evidence the app.log log4j log is being built at any time. I have a log4j.properties file in the Message Broker Java Project. However, I cannot find the log4j.properties file on the Broker. Where should the log4j.properties file be located?
Q2.) How do I control the file size of log4j so it rolls and does not fill up the file system?
The second part of the question is:
Q1.) How do I configure the broker wide to use a single log4j implementation across all Execution Groups?
Q2.) Do I use a Class Loader Configurable Service?
Q3.) How do I configure the Message Broker - Java Project to use the log4j Class Loader Configurable Service |
|
| Back to top |
|
 |
| sandman147 |
| Posted: Thu Mar 13, 2014 11:26 am Post subject: |
|
|
Apprentice
Joined: 01 Sep 2013
Posts: 42
|
Make sure your executable jar files necessary to log data is stored in the appropriate directory. In my case: C:\Program FIles\IBM\MQSI\7.0\jplugin
Provide brokerlog.xml path in your error handler routine so you can see the logs in the specified directory. To log messages you must call the function from with your esql code. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Mar 14, 2014 12:28 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Hi Eric,
If you can, switch from log4j to java.util.logging !!
This is what best practice tells us to use, and it is present in all JVM's since Java 3.1 (with Lumberjack), and above.
Alternatively there is a commons library that should allow you to use any code against any implementation, i.e. code with log4j but implement through java.util.logging...
That is, if you can have the developers upgrade the code to java.util.logging, have them do it! 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| stoney |
| Posted: Fri Mar 14, 2014 3:33 am Post subject: |
|
|
Centurion
Joined: 03 Apr 2013
Posts: 140
|
Q1.) How do I configure the broker wide to use a single log4j implementation across all Execution Groups?
Place the JAR file(s) in the shared-classes directory in the broker work path. There are also shared-classes directories for specific brokers and execution groups if you don't want to make those JAR files available to all brokers on that system.
Q2.) Do I use a Class Loader Configurable Service?
No need - any JAR files placed in the shared-classes are always available.
Q3.) How do I configure the Message Broker - Java Project to use the log4j Class Loader Configurable Service
When modifying the Java build path of the Java project, on the Libraries tab click on "Add External JARs..." and specify the path to the JARs in question. Do not select "Add JARs..." and add JAR files from your workspace, as this will end up including those JAR files in your BAR file. |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Fri Mar 14, 2014 5:26 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
| fjb_saper wrote: |
Hi Eric,
If you can, switch from log4j to java.util.logging !!
This is what best practice tells us to use, and it is present in all JVM's since Java 3.1 (with Lumberjack), and above.
|
Do you have a link for that "Best Practice" you speak of?
I'm collecting such info... |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Mar 14, 2014 9:50 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| JosephGramig wrote: |
Do you have a link for that "Best Practice" you speak of?
I'm collecting such info... |
Look up the posts from mqjeff on the log4j subject... they were pretty eloquent to me...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Fri Mar 14, 2014 10:17 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
I did not know eloquent meant "snarky" 
OK, if mqjeff said it, then it must be a good idea (and I agree that it is a good idea). Just trying to footnote and reference so BS types don't go "And who said that was a good idea?" without knowing either way.
Unfortunate that most of the "Best Practices" documents have aged and sections deprecated due to changes in the product. |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Fri Mar 14, 2014 10:24 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| JosephGramig wrote: |
I did not know eloquent meant "snarky"
OK, if mqjeff said it, then it must be a good idea (and I agree that it is a good idea). Just trying to footnote and reference so BS types don't go "And who said that was a good idea?" without knowing either way.
Unfortunate that most of the "Best Practices" documents have aged and sections deprecated due to changes in the product. |
Best practices aren't always documented in a structural way. You're free to compile your own suite of best practices, based on your experiences, and have other developers comment and amend.
However, in this case the pros of using the built-in solution are quite clear, whoever mentioned them.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Mar 14, 2014 7:45 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Our Logging Best Practice is
NEVER EVER log anything in production to a flat file UNLESS the output ls going to be less than 10Mb per year.
This allows us to log events in some batch jobs but not for anyone to use log4j or its equivalent.
We already have boundary logging features in all our flows. By this I mean:-
1) Every Input to a flow
2) Every Output from a flow
3) Every Error (Throw etc) encountered in a flow
The data includes every message crossing the boundary and the output logging event also includes other relevant information to help support staff.
Is written to its respective queue where other flows log the data to relevant DB tables. Ok, for some transaction rates this is not possible but in our line of business this works perfectly.
Thus my 'best practice' works for me and my environment. It may well not be applicable to someone else's.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Mon Mar 17, 2014 6:58 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
Gralgrathor,
I agree that every institution needs to compile their own "Best Practices" from what is a available. This is what I'm doing. If it is possible, I might publish those results. |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Tue Mar 18, 2014 3:44 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| smdavies99 wrote: |
| NEVER EVER log anything in production to a flat file UNLESS the output ls going to be less than 10Mb per year.[/b] |
If you use a rolling file appender, the output might be max. 10MB over the course of a century. But that's not what you mean, right? Is there any reason for such limiting the logging-output (other than preventing overflow of system/application partitions)?
I agree with you about the use of databases, by the way. Makes error-detection, handling and resolution much easier to handle. Files are useful as long as you're in the debugging stage, I think, but not a robust solution for production platforms.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Tue Mar 18, 2014 3:45 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| JosephGramig wrote: |
| what is a available |
Well, you've got page after page of best practices right here, so make the best of it! 
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Mar 18, 2014 4:26 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Gralgrathor wrote: |
| smdavies99 wrote: |
| NEVER EVER log anything in production to a flat file UNLESS the output ls going to be less than 10Mb per year.[/b] |
If you use a rolling file appender, the output might be max. 10MB over the course of a century. But that's not what you mean, right? Is there any reason for such limiting the logging-output (other than preventing overflow of system/application partitions)? |
People log to the file system, because they believe that it's quick and safe.
The whole point of writing log records is to ensure that when a problem is *discovered*, that you have diagnostic data available to determine when the problem *occured*, how *big* a problem it was, and ideally what caused it.
I've never run into any kind of rolling log based on file systems that didn't routinely underestimate the amount of data needing to be stored in order to solve 80% of the problems that are encountered, because they assume that a fixed file size is a good enough guess.
Using a database instead of a flat file lets you expire records based on their AGE, rather than based on the size of the database table. This means you are 95% likely to have records available that cover the occurrence of a given problem, compared to about 60% of the time with flat file logging. And databases are much more able to handle storing large volumes of data.
In my personal experience.
Using a messaging system as a log storage, instead of using a flat file or a database, allows you to be much more flexible in how you process, categorize, and store log data. And the reliability and speed is at least on par with a database, if not more reliable and faster. And you can then consume messages and log to several databases based on need AND a flatfile AND send messages to your pagers and etc. all at once, if you use a pub/sub messaging model.
So I really really prefer Broker monitoring events for all application level logging, rather than anything that requires me to hand-instrument code. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Mar 18, 2014 6:40 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| mqjeff wrote: |
In my personal experience.
Using a messaging system as a log storage, instead of using a flat file or a database, allows you to be much more flexible in how you process, categorize, and store log data. |
Please tell us ignorant masses exactly how using a Queue (I am assuming that this is what you mean) as log storage would allow you to:-
1) Search for particular errors, say one instance in 100,000 messages on the Q?
2) perform joins between Input data and Error/Output records as an aide to falut finding and subsequent rectification
3) handle a Really HUGE queue that is about to bring the whole system to a grinding halt because the Queue was full and none of the message had expiry times set. (happened to me on an old legacy system only last year)
If I am barking up the wrong street then I apologize in advance but IMHO and from bitter experience using queues for any form of storage there the message do not have limited life is really bad practice.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Mar 18, 2014 9:09 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I didn't mean to be so unclear.
I did not mean "store log records on a queue".
I meant "send log records to a queue, and use an application to consume them and store them somewhere else, like a database".
I said "log storage" because I was originally going to say "log sink" and I thought that was worse, but I didn't think through the implications of "log storage" to find a better word choice. |
|
| Back to top |
|
|
|
|
