ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexGeneral IBM MQ SupportCentralized logging for MQ error logs and syslog on UNIX

Post new topicReply to topic
Centralized logging for MQ error logs and syslog on UNIX View previous topic :: View next topic
Author Message
gbaddeley
PostPosted: Sun Feb 25, 2018 7:22 pm Post subject: Centralized logging for MQ error logs and syslog on UNIX Reply with quote

Padawan

Joined: 25 Mar 2003
Posts: 1809
Location: Melbourne, Australia

Hi folks,

Does anyone know of any robust UNIX solutions that provide centralized logging for MQ error logs and syslog messages, in real-time? It needs an app that allows log messages to be filtered and watched while critical changes are in progress, or investigating issues across multiple systems. It also requires the ability to look at past logs, and do reporting / data crunching. Preferably easy to use, setup, administer, maintain, low s/w cost or no s/w cost.

This for MQ / IIB technical operations use. Not for monitoring, incident alerting or notification (we already have enterprise tools for that).
_________________
Glenn
Back to top
View user's profile Send private message
exerk
PostPosted: Mon Feb 26, 2018 12:39 am Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 5855

If using MQ V9.0.4 the logs can additionally be JSON formatted, which the likes of Splunk can parse. Not sure if that's of any help to you?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.

Back to top
View user's profile Send private message
gbaddeley
PostPosted: Mon Feb 26, 2018 6:34 pm Post subject: Reply with quote

Padawan

Joined: 25 Mar 2003
Posts: 1809
Location: Melbourne, Australia

https://www.ibm.com/developerworks/community/blogs/messaging/entry/Storing_and_searching_MQ_logs_in_Elasticsearch

Has anyone implemented MQ error logs -> Logstash -> Elasticsearch -> Kibana ?
_________________
Glenn
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Mon Feb 26, 2018 6:40 pm Post subject: Reply with quote

Padawan

Joined: 25 Mar 2003
Posts: 1809
Location: Melbourne, Australia

https://www.digitalocean.com/community/tutorials/how-to-centralize-logs-with-rsyslog-logstash-and-elasticsearch-on-ubuntu-14-04

To capture UNIX syslog messages from IIB, one can use rsyslog ( http://www.rsyslog.com/ ) to send over TCP to a remote rsyslog server, and then use Logstash.

Has anyone implemented this?
_________________
Glenn
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Mon Feb 26, 2018 6:45 pm Post subject: Reply with quote

Padawan

Joined: 25 Mar 2003
Posts: 1809
Location: Melbourne, Australia

exerk wrote:
If using MQ V9.0.4 the logs can additionally be JSON formatted, which the likes of Splunk can parse. Not sure if that's of any help to you?

FYI, here is the IBM developerWorks article - https://developer.ibm.com/messaging/2017/12/05/introducing-mq-error-logs-json-format/
_________________
Glenn
Back to top
View user's profile Send private message
exerk
PostPosted: Tue Feb 27, 2018 12:19 am Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 5855

gbaddeley wrote:
exerk wrote:
If using MQ V9.0.4 the logs can additionally be JSON formatted, which the likes of Splunk can parse. Not sure if that's of any help to you?

FYI, here is the IBM developerWorks article - https://developer.ibm.com/messaging/2017/12/05/introducing-mq-error-logs-json-format/

And comparing that with the first link you posted, I can now see why IBM decided to do it "at source"!

Thank you for that Logstash etc. link - I'm going to have to stop ignoring those eye-catcher emails
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.

Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Feb 27, 2018 6:01 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 24972
Location: Ohio, USA

gbaddeley wrote:
https://www.ibm.com/developerworks/community/blogs/messaging/entry/Storing_and_searching_MQ_logs_in_Elasticsearch

Has anyone implemented MQ error logs -> Logstash -> Elasticsearch -> Kibana ?




But we're starting out with ELK and it seems to be going ok for Oracle & WAS. MQ & IIB / ACE coming shortly......
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Tue Feb 27, 2018 4:48 pm Post subject: Reply with quote

Jedi Council

Joined: 15 May 2001
Posts: 7427

We are going Splunking later this year, I hope.

To date, MQ's highly formatted, not-single-line-entry logs have been challenging for our current corporate logging facility. As in, not worth the effort and whose got the time to make it work.

T.Rob's blog has a really in depth post only a few weeks ago where shares a lot of the Splunky stuff he did with MQ at a client.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
zpat
PostPosted: Wed Feb 28, 2018 3:33 am Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5645
Location: UK

Why don't IBM use their own messaging to pub/sub the error log? (In addition to any file based error logging).
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexGeneral IBM MQ SupportCentralized logging for MQ error logs and syslog on UNIX
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.