Tools Developed, Supported, Sponsored and/or Released by KoreLogic
Below are some of the open source tools that KoreLogic staff have developed. We have released these tools under various open source licenses. Please contact us if you need clarification for any particular tool.
FTimes - A system baselining and evidence collection tool.
The primary purpose of FTimes is to gather and/or develop topographical information and attributes about specified directories and files in a manner conducive to intrusion and forensic analysis. FTimes is a lightweight tool in the sense that it doesn't need to be "installed" on a given system to work on that system, it has a small footprint, and it provides only a command line interface. FTimes implements two general capabilities: file topography and string search. File topography is the process of mapping key file attributes and metadata for a given set of files and directories. String search is the process of digging through a set of files while looking for specific sequences of bytes. Respectively, these capabilities are referred to as map mode and dig mode. FTimes is written in C and ported to many popular platforms (e.g., FreeBSD, Linux, MacOS, and Windows) including some mobile platforms (e.g., Android and iOS).
Giles - A production rule system compiler.
A production rule system (or "engine" in Giles parlance) is a program that is typically used to provide some sort of artificial intelligence. Giles engines are particularly well-suited to being expert systems, multi-log analyzers, and behavior-detection systems.
Existing production rule systems are often expensive, difficult to use, and accessible only through custom interfaces that require additional programmer effort and may be available only for one or a few languages.
Giles takes a novel approach: it compiles a description of an engine into a schema for a database system. Databases created using this schema become complete, self-contained production rule systems.
This approach has a number of benefits:
Programmers can use existing, ubiquitous database access libraries, available in essentially any language, to work with these engines.
For products that already use a supported database, a Giles engine adds no new software dependencies.
By using a SQLite engine, it becomes easy to embed a production rule system inside a larger product.
The engines can use the underlying database's data safety guarantees, including transactions and data durability guarantees.
The engines can handle large amounts of data over long periods of time.
Download the current stable release here [sign], or visit the Giles Git page. Questions or comments on the project may be directed to firstname.lastname@example.org. [key]
IDS Filters [sign] - A tool for creating and managing highly tailored IDS filtering rules.
With IDS Filters you can easily tune IDS filters to greatly reduce the number of false-positives, raise the effectiveness of the IDS deployment, and improve efficiency of IDS analysts. The current release includes sample rules to eliminate the most common sources of false-positive noise in a typical internal network. Currently, only the Dragon IDS is supported, but the code can be adapted to support any other IDS solution having a sufficiently powerful filtering mechanism.
KL-EL - A lightweight, embeddable expression language.
The KoreLogic Expression Language Library (libklel) is a C library that provides a simple expression language that can be embedded in other programs. It does not implement a full programming language, but rather a simpler expression language called KL-EL that is designed to provide arithmetic and logic operations in situations where embedding a full programming language would be overkill. KL-EL can access functions and variables exported from the embedding program, and is statically and strongly typed, which helps ensure that expressions are valid before they are executed. The embedding API is easy to use, and the library itself is very small.
Download the current stable release here [sign], or visit the SourceForge project page. Questions or comments on the project may be directed to email@example.com. [key]
MASTIFF - A static analysis framework that automates the process of extracting key characteristics from a number of different file formats.
To ensure the framework remains flexible and extensible, a community-driven set of plug-ins is used to perform file analysis and data extraction. While originally designed to support malware, intrusion, and forensic analysis, the framework is well-suited to support a broader range of analytic needs. In a nutshell, MASTIFF allows analysts to focus on analysis rather than figuring out how to parse files.
Download the current stable release here [sign], or visit the MASTIFF Git page to access the development code tree. Questions or comments on the project may be directed to firstname.lastname@example.org. [key]
KLogTail [sign] [sign-key] - A log file tailing program.
KLogTail was inspired by Craig Rowland's logtail, and it was designed to work on a number of platforms (e.g., FreeBSD, Linux, MacOS, and Windows).
WebJob - A secure automation framework that can be used support an arbitrary number of "jobs".
WebJob supports highly distributed and/or repetitious activities (e.g., evidence collection, integrity monitoring, enterprise search, configuration management, compliance verification, automated analysis, password cracking, testing, etc.) using a secure client-server, pull-based model. WebJob clients periodically download one or more jobs (in the form of pre-compiled programs or scripts) and execute them. Any output produced by these jobs is automatically packaged up and sent to the WebJob server for archival and/or further processing.
John the Ripper Tools :
rockyou.chr[sign] rockyou-lanman.chr[sign] README
- Updated .chr files for use with the John the Ripper password cracking tool. John uses .chr files to inform its "smart brute force" mode, generating password candidates using the most frequent characters first (rather than "aaaaaaa", "aaaaaab", "aaaaaac", etc). A large online database was recently compromised and passwords leaked; these .chr files are built using this large, current sample. See the README for more information and usage notes.
Miscellaneous Tools :
ddp [md5] [sign] [sign-key]
- DDP, short for Dd-Delta-Patch, is a tool for diffing and patching equal-sized dd(1) images. This tool was written as a companion to dd(1), and its purpose is to reduce the amount of storage required to maintain multiple snapshots of a given partition or disk image. Note: Version 1.0.0.ds29 is a development release.
probe_summ [sign] - A perl script to summarize Linux firewall logs--show the top N ports being scanned, the top destination IPs being scanned, the top source IPs sending traffic which is being blocked, and the hours during which the most packets were dropped. Should work for logs from both ipchains and iptables Linux boxes; you may have to tweak some of the tunables depending on how your firewall rules are set up.
tallyho [md5] [sign] [sign-key] - TallyHo is a libpcap-based sniffer program that counts (or tallies) various traffic attributes such as protocol/port/type distributions and byte counts.
weblogicdecode [sign] - WebLogic servers in Cluster mode leak the IP addresses of the backend servers in their session cookies. For some versions of WebLogic, you can manipulate this cookie and trick the webserver into port-scanning the backend network for you. This script decodes a number of different WebLogic cookie formats (they have changed with different WL versions).
www-version-harvest [sign] - Given a starting URL, this script finds/guesses all webservers linked off that page, and grabs Server: tokens from each (webserver software and version information). Useful for "taking the temperature" of a project's mirror servers--if a large percentage of them are running old, vulnerable webserver versions, you should be more suspicious of the software you download from any of them. Especially if the project does not PGP-sign their releases.
yaas [sign] yaas.cfg.sample [sign] imap-login [sign] - Yet Another Auth-before-Something. Watches syslogs of imapd logins and dynamically adds/removes /etc/hosts.allow file entries to allow access to other services, such as smtp, sshd, etc. (You need to create /etc/hosts.deny entries with ALL for each service for which you are using yaas to build /etc/hosts.allow entries.) Also, a small shell script to manually log in over IMAP-SSL, if you want to "register" your IP in the hosts.allow file without actually using a mail client.
LoadLimit.pm [sign] - A proof-of-concept perl module to mitigate application-level denial of service attacks for perl CGIs. Call from within perl CGIs (with or without mod_perl) to check the system load against configurable thresholds, and returns an error if the server is overloaded, short-circuiting potentially expensive operations.
Dragon IDS Tools :
- A shell script that checks on the status of Dragon sensor, replicator, and policy-management processes and connections, and restarts Dragon if things do not look right. This has been sufficient to catch the error-states we've seen most often on our sensors. Suitable to be run from cron or via WebJob.
dbcat [sign] - A perl script which reads a Dragon .db binary format log file and spit out a dragon.log style text output (suitable to feed to alarmtool, SQL-import tools, etc), or print out any readable packet data, suitable for doing tail -f dragon.db | dbgrep "something" | dbcat to watch sessions in real-time.
dbgrep [sign] - A perl script which reads a Dragon .db binary format log file, applies filters (similar to grep, or the Dragon 'drep' tool) and spits out a filtered .db binary format log file. Capable of filtering on source or dest IP or port, eventname, sensorname, or by regular expression in the bodies of captured packets. Supports following the TCP session of any event printed (such as following the session for all FTP:USER-ROOT events, rather than just one session at a time using Dragon's mksession tool). Supports negative filtering as well (-v, similar to grep). Since dbgrep's output is also a binary .db file, any number of dbgrep commands can be chained together to achieve an arbitrarily complex result.
decode_contentidpost [sign] - A perl script which reads the uploads from MarketScore spyware and decodes the contents, so you can easily review what potentially sensitive information has been uploaded to the mothership. Usage details in the script; requires both dbgrep and dbcat.
extract_web-get-exes [sign] - A perl script which reads WEB:GET-EXE events (a signature looking for any GET of an .EXE file; see MISC_LIGHT.lib) and extracts the Host: header and the full file path being requested. This can be used to profile your network and look for anomalous downloads. Windows networks will make many WEB:GET-EXE requests to Microsoft Windows Update servers, and there will likely be some regular downloads from dell.com, antivirus updates, and other presumably legitemate, reputable sites. But the remainder is very likely to be malware or spyware-infected machines, phoning home, downloading additional malware, etc. Usage details in the script; requires both dbgrep and dbcat.
Dragon IDS Signatures :
IRCBDCHAN.lib [sign] - This group contains custom IRC-backdoor-trojan Dragon NIDS signatures. These signatures detect clients registering with a number of IRC channel names commonly used as the remote-control channel for trojans/worms which register themselves after infection to await further instruction, report on progress compromising more machines, etc. NOTE they currently use port 6667, the default IRC port. It would be better to define a local COMPLEX port entry, such as 'I', which listed a wider range of ports commonly used by IRC servers, such as perhaps 6666-7000.
IRCBDCMDS.lib [sign] - This group contains custom IRC-backdoor-trojan Dragon NIDS signatures. These signatures detect the types of command strings, or phone-home reporting progress compromised machines, etc. NOTE they currently use port 6667, the default IRC port. It would be better to define a local COMPLEX port entry, such as 'I', which listed a wider range of ports commonly used by IRC servers, such as perhaps 6666-7000.
MISC_HUNGRY.lib [sign] - This group contains a few custom Dragon NIDS signatures which are known to be CPU-hungry. They should not be enabled on heavily loaded sensors if possible.
MISC_LIGHT.lib [sign] - This group contains a few custom Dragon NIDS signatures which should be fairly low-CPU-cost, and high-value.
ORACLE.lib [sign] - These sigs detect unusual, possibly malicious Oracle database queries/traffic. NOTE they currently use port 1521. That is the default port for Oracle's main listener, but it might easily have been changed by local DBAs. It would be better to define a local COMPLEX port entry, such as 'O', which listed all ports used by Oracle servers within your organization.
SPYWARE.lib [sign] - This group contains Dragon signatures for various known spyware / malware websites and tools.
SQL.lib [sign] - This group contains Dragon signatures for various SQL-injection attacks in HTTP requests, particularly in POST bodies. For the most part these should never triger on legitimate traffic, although handful of false-positives may be triggered by exceptionally insecure web applications.
WEBLOGIC.lib [sign] - These sigs detect unusual, possibly malicious WebLogic application server queries/traffic. NOTE: This library currently uses the 'W' COMPLEX rule to watch for traffic on known Web ports. However, this is sub-optimal: often WebLogic servers run on alternate ports such as 5000, 7001, etc, so these signatures will not monitor the right traffic. Some of these signatures are likely to false-positive on generic Web traffic, whereas if they were watching purely WebLogic sessions, their false positive rate would be very low; those signatures are currently disabled (re-enable them only after assigning them to a WebLogic-specific COMPLEX rule).