LogDistiller started with a simple observation concerning error log files: in real life, a single problem can produce a lot of traces. Even if some problems are perfectly identified, you rapidly get submerged by their traces and are unable to find your way between known and unknown traces. That leads to a common situation where error log files are simply ignored.
The purpose of LogDistiller is to continue to find useful information in error log files, by extracting known traces to let you concentrate on unknown ones. Each time you find similar traces indicating a specific new problem, you write a pattern matching these traces. Next time, LogDistiller will extract and count them from log files: you can now concentrate on other unknown traces.
Of course, nothing limits LogDistiller's usage to error-only log files: pattern recognition in traces can be applied to any sort of log files. You just need to know what traces you want to group (and a parser adapted to your log type must exist, of course).
LogDistiller is a log files merge and sort tool. It reads log files, parses them into structured log events with attributes, then classify them according rules configured in an XML file. Classification results go into reports, published according to the rules configuration: simply stored in a file, sent by mail, or even added in a news feed.
Some log parsers are included (syslog, Weblogic, simple line log, ... see the full list), but LogDistiller log files parsing is designed to be easy to extend: see LogTypes Extensions Center for example.
LogDistiller can be used in three ways:
LogDistiller is free open-source software; use it under the terms of the Apache License v2 license.
| - added LogEventBuilder helper class to ease parser writing - improvement: avoid empty sampling log files creation by sampling plugin - upgraded to Java 5 - added context information in warning messages - migrated from old CVS to git |
|
| November 29, 2009 | version 1.1 released extensions: - added stat plugin - added commonly used HTTP web server log format log types - added Siebel log format log types - moved csv log type to an extension to avoid licensing issue core: - feature: added "attributes" parameter to freq plugin, which should now be used instead of old "attribute", "attribute2", "attribute3" and "regexp" parameter (extended attributes are now here to replace "regexp") - changed license to Apache License v2 - feature: added extended attributes, lazily calculated as regexp on provided attributes, - added condition tagging support to give more plugin possibilities, - feature: (2783217) added compress'ed (*.Z) log files support in Ant tasks, - feature: added a Modello descriptor for LogDistiller classification rules, that gives us another form of documentation and a XML Schema equivalent to the current DTD and its documentation, - usability: (2671317) added a warning message if multiple values are set for a parameter |
| January 9, 2009 | version 1.0 released - created LogDistiller Extensions Center, to put code for LogDistiller extensions, - LogDistiller is now available in Maven central repository: <groupId>net.sf.logdistiller</groupId><artifactId>logdistiller</artifactId>, - created LogEventTestCase class to ease creating tests when writing a custom log parser, - opened a user mailing list, - added a menu bar to the GUI, - feature: improved regexp group handling in freq plugin: if negative, all groups defined in the regexp are concatenated |
| December 23, 2006 | version 0.9 released - reworked the documentation with a new quickstart guide, - improved the GUI with a progress bar, - change: Java's regex engine is now used instead of Jakarta ORO, - feature: added JBoss server log parser - improved the default classification rules generated by the GUI to get a more useful default report, - change: LogDistiller is now built with Maven 2 and Java 1.4 is mandatory (Java 1.3 not supported any more), - change: extracted log sampling features from freq plugin to a new sampling plugin. - feature: added CSV log parser, thanks to OstermillerUtils - feature: added Oracle's alert.log log parser - feature: added date.format and date.locale optional parameters to Weblogic log parser to change date format |
| December 6, 2005 | ERRATUM: sample custom log parser doesn't compile (forgot to add logSource):
please download erratum version 0.8 released - feature: added attribute2 and attribute3 parameters to freq plugin, to calculate freq on the concatenation of multiple attributes - feature: added logSource attribute to every log type (every parser has to be updated) - feature: added compressed="no|gzip|zip" to Ant logdistiller task to read logs directly from compressed archives - feature: added XML report format (format="xml") - improved text report format to get better legibility - feature: added extensible report format system - sample: added to the sample/ directory a custom log parser, to show how to manage a specific log format not initially provided by LogDistiller (in this case, a specific log4j conversion pattern) |
| October 24, 2005 | version 0.7 released - feature: added Log4J XML layout log parser - feature: added sampling feature to freq plugin (and other minor params): with sampling, a limited number of log events are saved for each attribute's value. - feature: added maxSave.count and maxSave.size (in kB) params on group definition and output definition (to set a default value for every group) to limit the amount of log events saved |
| October 13, 2005 | version 0.7RC released - added feed report publisher (using Rome) - update: added extensible report publishers, to be able to add more publishers than only file or mail - incompatible change: extension mechanism to add custom log types or plugins has been totally rewritten - update: replaced Ant task's logtype element with equivalent output element in rules file - update: added WARNING messages in global report |
| February 27, 2005 | version 0.6.1 released - feature: added the "new" dialog box to the GUI to create a new classification rules file - feature: added "batch" button in the GUI to generate Ant build file content for batch usage - update: renamed Ant's factory attribute with logtype for global consistency |
| February 20, 2005 | version 0.6 released - feature: added a plugin system to extend LogDistiller's handling of event groups - feature: added freq plugin, to calculate frequency of parameter values - feature: added a GUI to make interactive tests of rules configuration (thanks to JGoodies forms and Form Layout Maker) |
| February 5, 2005 | version 0.5.1 released - bugfix: logdistiller-logtypes.properties was not included in logdistiller.jar - feature: added DTD documentation (thanks to DTDDoc) |
| January 11, 2005 | version 0.5 released - greatly improved documentation, with addition of a sample directory - various simplifications in LogDistiller's usage |
| December 26, 2004 | site documentation is now generated with Maven 1 |
| July 29, 2004 | version 0.4 released - added logs.url parameter to point to report's publication website - and assemble groups into categories |
| June 20, 2004 | version 0.3 released now logdistillation can be run with Ant, which eases its usage a lot. |