LogDistiller started with a simple observation concerning error log files: in real life, a single problem can produce a lot of traces. Even if some problems are perfectly identified, you rapidly get submerged by their traces and are unable to find your way between known and unkown traces. That leads to a common situation where error log files are simply ignored.
The purpose of LogDistiller is to continue to find useful information in error log files, by extracting known traces to let you concentrate on unknown ones. Each time you find similar traces indicating a specific new problem, you write a pattern matching these traces. Next time, LogDistiller will extract and count them from log files: you can now concentrate on other unknown traces.
Of course, nothing limits LogDistiller's usage to error-only log files: pattern recognition in traces can be applyed to any sort of log files. You just need to know what traces you want to group (and a parser adapted to your log type must exist, of course).
LogDistiller is a log files merge and sort tool. It reads log files, parses them into structured log events with attributes, then classify them according rules configured in an XML file. Classification results go into reports, which are published according to the rules configuration: simply stored in a file, sent by mail, or even added in a news feed.
Some log parsers are included (syslog, Weblogic, simple line log, ... see the full list), but LogDistiller log files parsing is designed to be easy to extend.
LogDistiller can be used in three ways:
LogDistiller is free open-source software; use it under the terms of the GPL license.
|
- created LogDistiller Extensions Sandbox,
to put early-stage code for new extensions, - LogDistiller is now available in Maven central repository: <groupId>net.sf.logdistiller</groupId><artifactId>logdistiller</artifactId>,- created LogEventTestCase class to ease
creating tests when writing a custom log parser,- opened a user mailing list, - added a menu bar to the GUI, - feature: improved regexp group handling in freq plugin: if negative, all groups defined in the regexp are concatenated |
|
| December 23, 2006 | version 0.9 released - reworked the documentation with a new quickstart guide, - improved the GUI with a progress bar, - change: Java's regex engine is now used instead of Jakarta ORO, - feature: added JBoss server log parser - improved the default classification rules generated by the GUI to get a more useful default report, - change: LogDistiller is now built with Maven2 and Java 1.4 is mandatory (Java 1.3 not supported any more), - change: extracted log sampling features from freq plugin to a new sampling plugin. - feature: added CSV log parser, thanks to OstermillerUtils - feature: added Oracle's alert.log log parser - feature: added date.format and date.locale optional parameters to
Weblogic log parser to change
date format
|
| December 6, 2005 | ERRATUM: sample custom log parser doesn't compile (forgot to add logSource):
please download erratumversion 0.8 released - feature: added attribute2 and attribute3 parameters to
freq plugin, to calculate freq on
the concatenation of multiple attributes- feature: added logSource attribute to every log type (every parser has to be updated)- feature: added compressed="no|gzip|zip" to Ant logdistiller task to read
logs directly from compressed archives- feature: added XML report format ( format="xml")- improved text report format to get better legibility - feature: added extensible report format system - sample: added to the sample/ directory a custom log parser, to show how to manage a specific log format
not initially provided by LogDistiller (in this case, a specific log4j conversion pattern)
|
| October 24, 2005 | version 0.7 released - feature: added Log4J XML layout log parser - feature: added sampling feature to freq plugin (and other minor params): with sampling, a limited number of log events are saved for each attribute's value. - feature: added maxSave.count and maxSave.size (in kB) params on group
definition and output definition (to set a default value for every group) to limit the amount of log events
saved
|
| October 13, 2005 | version 0.7RC released - added feed report
publisher (using Rome)- update: added extensible report publishers, to be able to add more publishers than only file or mail - incompatible change: extension mechanism to add custom log types or plugins has been totally rewritten - update: replaced Ant task's logtype element with equivalent
output element in rules file- update: added WARNING messages in global report |
| February 27, 2005 | version 0.6.1 released - feature: added the "new" dialog box to the GUI to create a new classification rules file - feature: added "batch" button in the GUI to generate Ant build file content for batch usage - update: renamed Ant's factory attribute with logtype
for global coherence
|
| February 20, 2005 | version 0.6 released - feature: added a plugin system to extend LogDistiller's handling of event groups - feature: added freq plugin, to calculate frequency of parameter values - feature: added a GUI to make interactive tests of rules configuration (thanks to JGoodies forms and Form Layout Maker) |
| February 5, 2005 | version 0.5.1 released - bugfix: logdistiller-logtypes.properties was not included in
logdistiller.jar- feature: added DTD documentation (thanks to DTDDoc) |
| January 11, 2005 | version 0.5 released - greatly improved documentation, with addition of a sample directory- various simplifications in LogDistiller's usage |
| December 26, 2004 | site documentation is now generated with Maven |
| July 29, 2004 | version 0.4 released - added logs.url parameter to point to report's publication website - and assemble groups into categories |
| June 20, 2004 | version 0.3 released now logdistillation can be run with Ant, which eases its usage a lot. |