Robot
/
lnav
mirror of https://github.com/tstack/lnav.git
2
0
Fork 0
Code Issues Releases Wiki Activity
2,694 Commits 3 Branches 36 Tags 26 MiB
Commit Graph

56 Commits

Author SHA1 Message Date
Tim Stack d940d9d5eb [logfile] plumb gzip header through file meta 2022-09-20 20:07:44 -07:00
Tim Stack 740b827901 [logfile] handle logs with ANSI escapes 
This is a partial fix for handling ANSI escapes in
parts of log messages that would prevent regexes
from matching.  Still more work to do.

Related to #1057
 2022-09-14 21:43:36 -07:00
Tim Stack 66ec8e553e [build] fix static var issue 2022-09-04 07:13:32 -07:00
Tim Stack 13f161d21e [build] fix some warnings 2022-08-12 22:07:33 -07:00
Tim Stack 1ae1600f7b [line_buffer] cache compressed files 2022-08-01 14:56:48 -07:00
Timothy Stack 8ef581177c [line_buffer] add some performance counters for tracking SQL perf 
Also, check for keyboard input during SQL execution
so we can cancel.

Fixes #894
 2022-07-27 21:49:14 -07:00
Timothy Stack 39763b2d31 [vtab] add indexes for text columns 2022-07-10 21:00:45 -07:00
Timothy Stack 36e9433891 [perf] try to overlap line buffer i/o 2022-07-07 10:05:06 -07:00
Timothy Stack f9f797fc9d [logfile_sub_source] add log message watch expressions 
Fixes #539
 2022-06-28 22:23:56 -07:00
Timothy Stack a27198e8ca [regex101] add an integration with regex101 
... and a pile of other changes
 2022-04-30 13:06:01 -07:00
Timothy Stack 0785a432fa [console] colorize console output 2022-04-12 16:07:13 -07:00
Timothy Stack 81e63784e7 [modernize] some more cleanups 2022-03-31 08:59:33 -07:00
Timothy Stack 66ef5fdae1 [clang-format] init 2022-03-16 15:38:08 -07:00
Timothy Stack e593bde875 [build] fix some issues on xenial32 2021-03-30 22:50:10 -07:00
Timothy Stack f5e88b7158 [docs] add ARCHITECTURE.md and various other things 2021-02-13 12:41:48 -08:00
Phil Hord f10a303f54 Revert "Don't try to read concatenated gzip streams" 
This reverts commit 1d127053e9.
 2020-12-13 13:17:21 -08:00
Timothy Stack 51d1204a96 [arc] add content hash to archive temp dir and tech debt 2020-11-17 10:04:23 -08:00
Timothy Stack db8a3c4d38 [debt] doing some cleanup 
Defect Number:
    Reviewed By:
   Testing Done:
 2020-11-09 22:17:17 -08:00
Phil Hord 1d127053e9 Don't try to read concatenated gzip streams 
Don't try to continue reading the next stream of a concatenated
gzip file.  The next stream may be CRC noise or other garbage.

Maybe in the future we should look for a gzip header in the
following bytes of the stream and try to decode from there.
But it's not clear that anyone ever uses this supposed gzip
feature anyway.

Let's just end the stream when we reach EOS. Also, if the
stream fails to init, let's leave it closed instead of throwing
an error no one is likely to catch.  Log the error msg from
zlib if one is provided.
 2020-10-19 20:49:36 -07:00
Timothy Stack ce0383dee1 [build] fix some warnings 2020-09-11 23:10:11 -07:00
Timothy Stack 0940e97b2b [nits] fix some naming issues 
Fixes #169
 2020-08-30 22:13:56 -07:00
Timothy Stack 138a506b1a [perf] a few performance tweaks 2020-04-23 06:42:48 -07:00
Phil Hord 5513deeade Add a gzip indexing class for faster gz navigation 
The gzread function is slow.  Every time you seek to a new location, the
whole file up to that position has to be decompressed again.  This causes
massive lags when trying to do simple things in lnav on a large .gz file.

Use the zlib inflate* functions instead and record the dictionary
periodically while processing the file the first time.  Then use
inflateSetDictionary to restore the dictionary to a convenient
location when trying to seek into the file again in the future.

Use a default period of 1MB of compressed data for syncpoints.
Each syncpoint uses 32KB. This is a ratio of 3.2%.  For example,
a 1GB .gz file (compressed size) will require us to keep 32MB
of index data in memory. A better method may be to use a fixed
number of syncpoints and divide the file appropriately. This
would keep the memory bounded at the cost of slower file
navigation on large .gz files.

Use pread to read the data for the stream decompressor and remove
the lock_hack previously employed.

NB. The documentation on these zlib functions is sparse. I followed
the example in zlib/examples/zran.c, but I used the z_stream total_in
and total_out variables instead of keeping my own separately as zran.c
does.  Maybe this is incompatible with some very old zlib versions.
I haven't looked.
 2019-08-17 18:40:55 -07:00
Timothy Stack 02c53976b4 Some refactoring for the line buffer and other cleanups 
Fixes #649
 2019-06-15 06:32:02 -07:00
Timothy Stack fb7d6eafaf [build] some refactoring of files 2019-05-08 05:30:59 -07:00
Timothy Stack 66ef968529 [logfile] fix a performance bug in searching json logs 2018-12-16 21:45:10 -08:00
Timothy Stack 2589345e5c [perf] improve initial indexing times 2018-10-17 07:03:33 -07:00
Phil Hord 5a20829ed4 dynamic exception specifications are deprecated in C++11 
Remove exception specifications to appease modern compilers and reduce
warning counts.

I sort of hate to remove all these exception-decls, but they are in fact
deprecated, and my compiler loves to complain.
 2018-03-29 10:43:44 -07:00
Timothy Stack 88a192aeeb [line_buffer] if the size of the file is known, reduce the requested buffer size and ignore attempts to shrink the buffer 2018-03-25 23:36:18 -07:00
Timothy Stack 34d1422b76 [highlight] only highlight keywords in text files 
Fixes #408
 2017-02-18 22:35:18 -08:00
Timothy Stack 99eb3a54f3 [spectro] support marking lines in a bucket 2016-03-22 23:19:26 -07:00
Timothy Stack e8f400abfa [logfile] truncation detection was not taking into account compressed files 2016-03-21 21:10:52 -07:00
Timothy Stack 6e196bff79 [line_buffer] clear cached data when doing a read 2015-12-22 22:44:42 -08:00
Timothy Stack b6e10f4ee4 bump the max line size to try and solve some json log issues 2015-11-29 20:35:13 -08:00
Timothy Stack 25a375f08d [misc] fix various bugs 2015-08-19 21:02:26 -07:00
Timothy Stack 5a7c5a2139 [piper] the piper was not exiting if the stream it was reading did not terminate with a line feed 2015-04-21 22:25:54 -07:00
Timothy Stack cfa628f6c4 various fixes 
Defect Number:
    Reviewed By:
   Testing Done:
 2014-11-11 21:18:55 -08:00
Timothy Stack a71ca1f1ca [filter] some improvements to the core filtering code 2014-10-19 22:16:40 -07:00
Timothy Stack 9ff2d3774d [line_buffer] split long lines 
Fixes #81
 2014-03-15 04:40:58 -07:00
Timothy Stack b6baae7730 [debug] more debugging stuff 
Defect Number:
    Reviewed By:
   Testing Done:
 2014-03-06 06:58:49 -08:00
Timothy Stack 601c19a537 [scrollbar] change the look of the scrollbar 2014-02-20 20:01:06 -08:00
Timothy Stack 6a38619ac7 [piper] fix for partial lines when reading stdin 2014-02-19 05:58:31 -08:00
Timothy Stack 5ef8a168a7 [perf] reduce copies of log messages 2014-02-01 06:41:11 -08:00
Timothy Stack 2d33cb7f7e [log format] some more log format work 2013-06-29 11:00:34 -07:00
Timothy Stack 865849f017 [spam] uncrustify pass 2013-05-27 21:35:00 -07:00
Timothy Stack f62b5c271c [release] bump to 0.5.1 2013-05-02 23:02:03 -07:00
Timothy Stack a92785e889 [piper_proc] add timestamp and file write options for stdin 2013-04-30 21:48:16 -07:00
Timothy Stack d4f170e747 cppcheck fixes 2013-04-17 09:27:12 -07:00
Timothy Stack c4e9f58bf0 [bugs] fix some long-standing bugs 
* grep_proc.cc: When a request is queued with the start line
   == -1, we need to start searching from the highest line
   ever seen and not the last line processed.
 * line_buffer.cc: If a partial line was read, we need to
   avoid returning another line if more data is appended
   to the file.
 * lnav.cc: Accept file name patterns on the command-line that
   don't match any files yet.  Initialize the screen before
   redirecting stderr to the log file or /dev/null.
 * log_format.hh: Add some comments.  Start to add back support
   for scrubbing.
 * logfile_sub_source.cc: Move scrubbing to the format impl.
 * textview_curses.hh: Add comments.
 2012-09-22 16:15:15 -07:00
Timothy Stack b68a1bba9b [test] some cleanup for tests and other things 
Just a cleanup pass
 2012-06-05 13:18:59 -07:00