3.1 KiB
Architecture
This document covers the internal architecture of the Logfile Navigator (lnav), a terminal-based tool for viewing and analyzing log files.
Goals
The following goals drive the design and implementation of lnav:
-
Don't make the user do something that can be done automatically.
Example: Automatically detect log formats for files instead of making them specify the format for each file.
-
Be performant on low-spec hardware.
Example: Prefer single-threaded optimizations over trying to parallelize
-
Operations should be "live" and not block the user from continuing to work.
Example: Searches are run in the background.
-
Provide context-sensitive help.
Example: When the cursor is over a SQL keyword/function, the help text for that is shown above.
-
Show a preview of operations so the user knows what is going to happen.
Example: When entering a
:filter-outcommand, the matched parts of the lines are highlighted in red.
Overview
The whole of lnav consists of a log file parser, text UI, integrations with SQLite, command-line interface, and commands for operating on logs. Since the majority of lnav's operations center around logs, the core data-structure is the combined log message index. The message index is populated when new messages are read from log files. The text UI displays a subset of messages from the index. The SQLite virtual-tables allow for programmatic access to the messages and lnav's internal state.
File Monitoring
Each file being monitored by lnav has an associated logfile
object, be they plaintext files or files with a recognized format. These
objects are periodically polled by the main event loop to check if the file
was deleted, truncated, or new lines added. While reading new lines, if no
log format has matched yet, each line will be passed through the log format
regular expressions to try and find a match. Each line that is read is added
to an index
Why is mmap() not used?
Note that file contents are consumed using pread(2)/read(2) and not
mmap(2) since mmap(2) does not react well to files changing out from
underneath it. For example, a truncated file would likely result in a
SIGBUS.
Log Messages
As files are being indexed, if a matching format is found, the file is "promoted" from a plaintext file to a log file. When the file is promoted, it is added to the logfile_sub_source, which collates all log messages together into a single index.
User Interface
The lnav text-user-interface is built on top of the basic drawing functionality of ncurses. However,