ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Log4j support pack does not roll

Post new topic  Reply to topic Goto page 1, 2  Next
 Log4j support pack does not roll « View previous topic :: View next topic » 
Author Message
souciance
PostPosted: Tue Jan 09, 2018 10:28 pm    Post subject: Log4j support pack does not roll Reply with quote

Disciple

Joined: 29 Jun 2010
Posts: 169

Hello,

We are using IIB 10 and we have built a shared library subflow which uses the log4j support pack to write logs via log4j. So far so good.

This shared library is deployed to multiple integration servers.

In our brokerlog.xml we have multiple appenders where each appender writes to a seperate file. This so each IIB app generates its own file. The appenders are of type rollingfileappender or dailyfileappender.

What we have noticed is that the appenders do not roll. After some googling I am suspecting that this is either because it is a shared library or because multiple dataflowengine.exe processes are referring to log4j.

Either way something is preventing the logs from rolling. In fact even if I stop the integration servers where the libraries are deployed I still cannot deploy the log files. This is because it seems since all integration servers are denoted dataflowengine.exe it cannot tell which is which so to delete log files manually you have to stop all integration servers.

Has anyone experienced this and solved it? Do you use static libraries instead? I would think coming to version 10 IIB would provide its own built in logging mechanism on par at least with log4j if not better that works painlessly.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Jan 10, 2018 5:45 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

You should not have multiple integration servers (e.g.) log to the same file. This is like having multiple JVMs logging to the same file. Make the integration server name part of the log file name... just to be safe.

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
souciance
PostPosted: Wed Jan 10, 2018 6:10 am    Post subject: Reply with quote

Disciple

Joined: 29 Jun 2010
Posts: 169

fjb_saper wrote:
You should not have multiple integration servers (e.g.) log to the same file. This is like having multiple JVMs logging to the same file. Make the integration server name part of the log file name... just to be safe.

Have fun


Hi,

Multiple integration servers don't log to the same file.

We have multiple integration servers --> multiple IIB apps --> shared library --> multiple log4j appenders --> one log file per IIB app.

So essentially one IIB app logs to its own app and the IIB app is deployed to its own integration server.
Back to top
View user's profile Send private message
souciance
PostPosted: Wed Jan 10, 2018 6:16 am    Post subject: Reply with quote

Disciple

Joined: 29 Jun 2010
Posts: 169

Here is example:


Code:
-<appender class="org.apache.log4j.RollingFileAppender"   name="IP8_MCHServices">

<param name="append" value="true"/>

<param name="maxFileSize" value="1MB"/>

<param name="maxBackupIndex" value="10"/>

<param name="file" value="F:/Log4j/logs/IP8/IP8_MCHServices.log"/>


-<layout class="org.apache.log4j.PatternLayout">

<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss}%x %-5p %t - %m%n "/>

</layout>

</appender>


-<appender class="org.apache.log4j.RollingFileAppender" name="IP8_DocumentServices">

<param name="append" value="true"/>

<param name="maxFileSize" value="1MB"/>

<param name="maxBackupIndex" value="10"/>

<param name="file" value="F:/Log4j/logs/IP8/IP8_DocumentServices.log"/>


-<layout class="org.apache.log4j.PatternLayout">

<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss}%x %-5p %t - %m%n "/>

</layout>

</appender>
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jan 10, 2018 6:28 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

souciance wrote:
I would think coming to version 10 IIB would provide its own built in logging mechanism on par at least with log4j if not better that works painlessly.


Well there are better java-based logging solutions than log4j these days (and that wretched support pac has it's own problems, as we found to our cost) so you might want to look through some of the discussions on logging we've had in here to see if something suits you better.

As to IIB, it's been the position of IBM for some time that IIB is a mid point and logging is better performed by the applications at each end. I'm personally unconvinced that's still the case in these modern days and I would ask you to consider raising an RFE that describes your requirements. It's my experience that a detailed RFE does better than a more fuzzy "IIB should have logging", and you should post the link here. I'd vote for it.

Finally, and assuming you're trying to log activity in the flows for troubleshooting or audit purposes, consider using the event messages to export the information and have a single flow writing to the log. This will help prevent the problem my worthy associate mentions. My 2 cents is that even though you have these deployed separately, the logging class is common and that's what's holding all these files open.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
souciance
PostPosted: Wed Jan 10, 2018 6:33 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Disciple

Joined: 29 Jun 2010
Posts: 169

Vitor wrote:
souciance wrote:
I would think coming to version 10 IIB would provide its own built in logging mechanism on par at least with log4j if not better that works painlessly.


Well there are better java-based logging solutions than log4j these days (and that wretched support pac has it's own problems, as we found to our cost) so you might want to look through some of the discussions on logging we've had in here to see if something suits you better.

As to IIB, it's been the position of IBM for some time that IIB is a mid point and logging is better performed by the applications at each end. I'm personally unconvinced that's still the case in these modern days and I would ask you to consider raising an RFE that describes your requirements. It's my experience that a detailed RFE does better than a more fuzzy "IIB should have logging", and you should post the link here. I'd vote for it.

Finally, and assuming you're trying to log activity in the flows for troubleshooting or audit purposes, consider using the event messages to export the information and have a single flow writing to the log. This will help prevent the problem my worthy associate mentions. My 2 cents is that even though you have these deployed separately, the logging class is common and that's what's holding all these files open.


Off course, but we using the one provided by IBM which we thought would save some time rather than write our own. Otherwise we are looking into moving away from this and logging to ELK stack instead which is by far much easier and better and saves us these issues.

I'll look into the RFE process, thanks.

Well we even have different appenders pointing to different files but I am still suspecting that they logging class is somehow locking all of them.
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jan 10, 2018 6:51 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

souciance wrote:
Off course, but we using the one provided by IBM which we thought would save some time rather than write our own.


With respect, a Cat 3 support pac like this is hardly "provided by IBM"

souciance wrote:
Otherwise we are looking into moving away from this and logging to ELK stack instead which is by far much easier and better and saves us these issues.


You don't mention which level of IIB 10 you're on, but there's an ELK solution in 10.0.0.7 - see Send IIB logs to a Logmet Kibana dashboard in Bluemix. Not a replacement for the log4j solution and not a direct fit for your requirement, but something you might want to consider using and extending.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jan 10, 2018 7:04 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

souciance wrote:
Off course, but we using the one provided by IBM which we thought would save some time rather than write our own.


In the interests of full transparency we thought the same. Then after about a year of issues, problems and gotchas we decided that the TCO would be smaller if we wrote our own log4j wrapper. This has survived to this day, and allowed us to switch seamlessly to java.util.logging (which according to the Java gnomes is faster, cheaper, and shinier).

We too are looking into ELK, but on a enterprise level so we'll be logging into whatever the enterprise uses rather than an IIB-specific solution. Should be in place by 2020 if the Enterprise Architecture people can be made to hurry..........
_________________
Honesty is the best policy.
Insanity is the best defence.


Last edited by Vitor on Wed Jan 10, 2018 7:20 am; edited 1 time in total
Back to top
View user's profile Send private message
souciance
PostPosted: Wed Jan 10, 2018 7:09 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Disciple

Joined: 29 Jun 2010
Posts: 169

Vitor wrote:
souciance wrote:
Off course, but we using the one provided by IBM which we thought would save some time rather than write our own.


In the interests of full transparency we thought the same. Then after about a year of issues, problems and gotchas we decided that the TCO would be smaller if we wrote our own log4j wrapper. This has survived to this day, and allowed us to switch seamlessly to java.util.logging (which according to the Java gnomes is faster, cheaper, and shinier).

We too are looking into ELK, but on a enterprise level so we'll be logging into whatever the enterprise uses rather than an IIB-specific solution. Should be in place by 2020 if the Enterprise Architecture people can be made to hurry..........


Cool, yeah the ELK stack looks awesome and with stats and alerts and with metrics from the hosts you can get a total monitoring of IIB and the host as well.

Our ideal scenario is to push IIB and newer systems to log to ELK so that if you search for an id say a purchase order id, you can trace it in ELK and see where it has ended and which systems the data has gone through. That will save us a lot of tickets related to "Where is my order...?"
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jan 10, 2018 7:23 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

souciance wrote:
Our ideal scenario is to push IIB and newer systems to log to ELK so that if you search for an id say a purchase order id, you can trace it in ELK and see where it has ended and which systems the data has gone through. That will save us a lot of tickets related to "Where is my order...?"


Exactly the scenario event messages in IIB fulfill.

Exactly how we broke the log4j support pac.

Exactly why we use IBM's BAM stack to do exactly this today.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
souciance
PostPosted: Wed Jan 10, 2018 7:46 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Disciple

Joined: 29 Jun 2010
Posts: 169

Vitor wrote:
souciance wrote:
Our ideal scenario is to push IIB and newer systems to log to ELK so that if you search for an id say a purchase order id, you can trace it in ELK and see where it has ended and which systems the data has gone through. That will save us a lot of tickets related to "Where is my order...?"


Exactly the scenario event messages in IIB fulfill.

Exactly how we broke the log4j support pac.

Exactly why we use IBM's BAM stack to do exactly this today.


Yeah but don't you need to use monitoring events for all the above?
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jan 10, 2018 8:05 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

souciance wrote:
Yeah but don't you need to use monitoring events for all the above?


Vitor wrote:
Exactly the scenario event messages in IIB fulfill.


Required some upfront engineering but it's fast, reliable and (most importantly) we can change the number and content of the messages administratively. Having designed the SDLC for IIB here, and the stage gates for getting it into Prod, it was a PITA when the only change was someone trying to add a new piece of information to the log message.

The only thing we use the Java log for is actually Info/Warning/Error messages from the business code inside IIB. And before anyone says it, yes I know you can use ESQL THROW et al to do this but using the Java allows us to format the application logs into a standard for our log scraping ticket writing software.

It's this part that we're looking to move into ELK; combining IIB system messages and application messages into a single searchable location with (if the plan comes together) all the WAS messages, Oracle messages and (if I have my way) coffee machine messages. No more mornings without coffee.....

The activity monitoring ("what happened to my order") will continue to be performed with event messages post-ELK via BAM. We like it, the users like it and management like it - it's easy to produce awesome graphs showing how many things we processed, how far under SLA we are and similar things that make management coo when part of a PowerPoint.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
souciance
PostPosted: Wed Jan 10, 2018 10:58 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Disciple

Joined: 29 Jun 2010
Posts: 169

Vitor wrote:
souciance wrote:
Yeah but don't you need to use monitoring events for all the above?


Vitor wrote:
Exactly the scenario event messages in IIB fulfill.


Required some upfront engineering but it's fast, reliable and (most importantly) we can change the number and content of the messages administratively. Having designed the SDLC for IIB here, and the stage gates for getting it into Prod, it was a PITA when the only change was someone trying to add a new piece of information to the log message.

The only thing we use the Java log for is actually Info/Warning/Error messages from the business code inside IIB. And before anyone says it, yes I know you can use ESQL THROW et al to do this but using the Java allows us to format the application logs into a standard for our log scraping ticket writing software.

It's this part that we're looking to move into ELK; combining IIB system messages and application messages into a single searchable location with (if the plan comes together) all the WAS messages, Oracle messages and (if I have my way) coffee machine messages. No more mornings without coffee.....

The activity monitoring ("what happened to my order") will continue to be performed with event messages post-ELK via BAM. We like it, the users like it and management like it - it's easy to produce awesome graphs showing how many things we processed, how far under SLA we are and similar things that make management coo when part of a PowerPoint.


Aha, yeah my current client does not use monitoring events and I think to set that up right now would be greater effort than move everything to ELK. Besides with ELK we can also go to the broader enterprise community and tell them to send their logs here too and get a better centralised logging.

Let's hope the next releases improve things but I suspect next features will focus more on cloudy things.
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jan 10, 2018 11:10 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

souciance wrote:
Aha, yeah my current client does not use monitoring events and I think to set that up right now would be greater effort than move everything to ELK. Besides with ELK we can also go to the broader enterprise community and tell them to send their logs here too and get a better centralised logging.


Whatever fits your needs on your site. I must say we've found it convenient (given the number of "orders" we're trying to track) to have them in a database than a log file (even a highly searchable ELK based one)

souciance wrote:
Let's hope the next releases improve things but I suspect next features will focus more on cloudy things.


Who needs hope when you have an RFE? The point is to steer IBM towards things that you want them to do.

Though cloud is the way everything is going. The likely ELK solution for us is the cloud based IBM one. There's no appetite for managing it in-house.

I repeat that you need to do what fits your needs on your site. I'm just trying to provide perspective and viewpoint.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
souciance
PostPosted: Thu Jan 11, 2018 5:50 am    Post subject: Re: Log4j support pack does not roll Reply with quote

Disciple

Joined: 29 Jun 2010
Posts: 169

Vitor wrote:
souciance wrote:
Aha, yeah my current client does not use monitoring events and I think to set that up right now would be greater effort than move everything to ELK. Besides with ELK we can also go to the broader enterprise community and tell them to send their logs here too and get a better centralised logging.


Whatever fits your needs on your site. I must say we've found it convenient (given the number of "orders" we're trying to track) to have them in a database than a log file (even a highly searchable ELK based one)

souciance wrote:
Let's hope the next releases improve things but I suspect next features will focus more on cloudy things.


Who needs hope when you have an RFE? The point is to steer IBM towards things that you want them to do.

Though cloud is the way everything is going. The likely ELK solution for us is the cloud based IBM one. There's no appetite for managing it in-house.

I repeat that you need to do what fits your needs on your site. I'm just trying to provide perspective and viewpoint.


I haven't tried so much the bluemix ELK but rather looking at logsense and logz.io.

Doesn't monitoring events have any impact on the performance of IIB?
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Log4j support pack does not roll
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.