[WNYLUG-Users] Centralized Logging Solution

Darin Perusich darin at darins.net
Sun Jul 31 20:01:10 EDT 2011


This is a good solution for a centralizing your logs, I use it to
archive in logs from all my servers and devices. The loganalyzer web
interface is nice but I find myself using grep more often and it can't
touch splunk for searching your logs. Let me make a couple
recommendations....

- Save the logs to files and the database. Files for long term storage
and the db for quick searches
- Do not store more than a few weeks worth of logs in the database,
like two. You can't compress this data and it will quickly get out of
control. Also the more data stored, the longer it'll take to query the
data and loganalyzer will time out.
- Put loganalyzer on a different server, and possible mysql as well.
This machine will be very busy and the less doing the more easily
it'll be able to keep up with the incoming logs.
- Install rsyslog wherever you can and use the REPL interface...you'll
never loose another log message again.
- Setup Splunk and configure it to slurp up ONLY the log files you
want so you stay under the free limit. I know you said you didn't want
to use splunk but it is the best way to search your logs hands down. I
run it on a dedicated server and nfs mount the log directory and have
configured with a whitelist of files to index.
- If you need to monitor your logs for things like abnormal login
attempts, etc. take a look at SEC - simple event correlator. Very
powerful tool for monitoring and alerting based on log abnormalities.
http://simple-evcorr.sourceforge.net/

--
Later,
Darin



On Sun, Jul 31, 2011 at 10:43 AM, Robert Wolfe <rwolfe at fpsoft.net> wrote:
> Well, decided I would go after a better log management and consolidation
> system that is better (and cheaper in the long run) than Splunk.
>
> What I was looking for was the ability to:
>
> 1) consolidate all logs files onto one central server,
> 2) store the logs on that central server using a MySQL backend,
> 3) have the ability to search that database with a web frontend, and
> 4) have that web front be a secure one secured with usernames and passwords
> at the very least.
>
> So, having said that, I have found three things that meet my requirements
> (MySQL being a given):
>
> 1) rsyslogd (http://www.rsyslog.com)
> 2) LogAnalyzer (http://loganalyzer.adiscon.com/)
> 3) MySQL (http://www.mysql.com)
>
> Note that 1 and 2 are made by the same company.
>
> Note that there is also an open source client for WIndows 32-bit and 64-bit
> machines that will send system event logs to a remote rsyslogd server:
>
> http://code.google.com/p/eventlog-to-syslog/
>
> So if you are looking for a decent logging solution, may I recommend giving
> this setup a try and it seems so far to be working well for me here at home.
>
> _______________________________________________
> Users mailing list
> Users at wnylug.org
> http://wnylug.org/mailman/listinfo/users_wnylug.org
>



More information about the Users mailing list