Robot
/
lnav
mirror of https://github.com/tstack/lnav.git
2
0
Fork 0
Code Issues Releases Wiki Activity

[sql] add a regexp_capture tablue-valued function 

Fixup the release makefile a bit.
This commit is contained in:
Timothy Stack 2017-03-15 17:01:58 -07:00
parent f6138662ae
commit c0684a97cc
26 changed files with 658 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
NEWS
View File

@ -32,6 +32,9 @@ lnav v0.8.2:
* Added a 'top_time' column to the lnav_views table so that you can get
the timestamp for the top line in views that are time-based as well as
allowing you to move the view to a given time with an UPDATE statement.
* Added a 'regexp_capture(<string>, <pattern>)' table-valued function for
getting detailed results from matching a regular expression against a
string.
Interface Changes:
* The color used for text colored via ":highlight" is now based on the

5
README
View File

@ -15,10 +15,11 @@ efficiently zero in on problems.
PREREQUISITES
-------------
Lnav requires the following software packages:
The following software packages are required to build lnav:
gcc/clang - A C++14-compatible compiler.
libpcre - The Perl Compatible Regular Expression (PCRE) library.
sqlite - The SQLite database engine.
sqlite - The SQLite database engine. Version 3.9.0 or higher is required.
ncurses - The ncurses text UI library.
readline - The readline line editing library.
zlib - The zlib compression library.

6
README.md
View File

@ -21,14 +21,16 @@ efficiently zero in on problems.
Prerequisites
-------------
Lnav requires the following software packages:
The following software packages are required to build lnav:
* gcc/clang - A C++14-compatible compiler.
* libpcre - The Perl Compatible Regular Expression (PCRE) library.
* sqlite - The SQLite database engine.
* sqlite - The SQLite database engine. Version 3.9.0 or higher is required.
* ncurses - The ncurses text UI library.
* readline - The readline line editing library.
* zlib - The zlib compression library.
* bz2 - The bzip2 compression library.
* libcurl - The cURL library for downloading files from URLs. Version 7.23.0 or higher is required.
Installation

2
configure.ac
View File

@ -139,7 +139,7 @@ AS_VAR_IF([ax_cv_curses],[yes],[],
AX_PATH_LIB_PCRE([], [AC_MSG_ERROR([pcre required to build])])
AX_PATH_LIB_READLINE
LNAV_WITH_SQLITE3("3.6.0")
LNAV_WITH_SQLITE3("3.9.0")
case "$host_os" in
*)

1
docs/source/sqlext.rst
View File

@ -129,7 +129,6 @@ Additional string comparison and manipulation functions:
algorithm as the *logline* table (see :ref:`data-ext`). The discovered
data is returned as a JSON-object that you can do further processing on.
File Paths
----------

7
docs/source/sqltab.rst
View File

@ -79,3 +79,10 @@ http_status_codes
The **http_status_codes** table is a handy reference that can be used to turn
HTTP status codes into human-readable messages.
regexp_capture(<string>, <regex>)
---------------------------------
The **regexp_capture()** table-valued function applies the regular expression
to the given string and returns detailed results for the captured portions of
the string.

16
release/Makefile
View File

@ -10,18 +10,16 @@ clean-outbox: outbox
rm -f outbox/*
PACKAGE_URLS = \
http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz \
http://ftp.gnu.org/gnu/automake/automake-1.15.tar.gz \
https://ftp.gnu.org/gnu/make/make-4.2.1.tar.gz \
ftp://invisible-island.net/ncurses/ncurses-5.9.tar.gz \
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz \
ftp://ftp.cwru.edu/pub/bash/readline-6.3.tar.gz \
http://zlib.net/zlib-1.2.8.tar.gz \
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.40.tar.gz \
https://ftp.gnu.org/gnu/readline/readline-6.3.tar.gz \
http://www.zlib.net/zlib-1.2.11.tar.gz \
http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz \
http://sqlite.org/2016/sqlite-autoconf-3130000.tar.gz \
http://openssl.skazkaforyou.com/source/openssl-1.0.1t.tar.gz \
http://www.libssh2.org/download/libssh2-1.7.0.tar.gz \
http://curl.haxx.se/download/curl-7.48.0.tar.gz
https://sqlite.org/2017/sqlite-autoconf-3170000.tar.gz \
https://www.openssl.org/source/openssl-1.0.2k.tar.gz \
https://www.libssh2.org/download/libssh2-1.8.0.tar.gz \
https://curl.haxx.se/download/curl-7.53.1.tar.gz
.PHONY: linux freebsd pkger download-pkgs

2
release/vagrant-static/Vagrantfile vendored
View File

@ -26,7 +26,7 @@ Vagrant.configure(2) do |config|
config.vm.define :linux do |linux|
config.vm.provision "shell", path:"pkg.sh"
config.vm.provision "shell", path:"provision.sh"
linux.vm.box = "hansode/centos-5.6-x86_64"
linux.vm.box = "centos/7"
end
config.vm.define :pkger do |pkger|

6
release/vagrant-static/build.sh
View File

@ -1,5 +1,9 @@
#! /usr/bin/env bash
if test x"${OS}" != x"FreeBSD"; then
source scl_source enable devtoolset-4
fi
FAKE_ROOT=/home/vagrant/fake.root
SRC_VERSION=$1
@ -33,8 +37,6 @@ OS=$(uname -s)
if test x"${OS}" != x"FreeBSD"; then
../lnav/configure \
LDFLAGS="-L${FAKE_ROOT}/lib" \
CC="gcc44" \
CXX="g++44" \
CPPFLAGS="-I${FAKE_ROOT}/include" \
PATH="${FAKE_ROOT}/bin:${PATH}"
else

16
release/vagrant-static/pkg.sh
View File

@ -2,12 +2,16 @@
OS=$(uname -s)
if test x"${OS}" != x"FreeBSD"; then
wget -N http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
sudo rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
sudo rpm -K rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
sudo rpm -U rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
sudo yum install -y m4 git gcc44-c++
sudo yum install -y \
m4 \
autoconf \
automake \
ncurses \
ncurses-devel \
git \
centos-release-scl \
perl-Data-Dumper
sudo yum install -y "devtoolset-4-gcc*"
else
pkg install -y wget git gcc m4
fi

23
release/vagrant-static/provision.sh
View File

@ -1,5 +1,9 @@
#! /usr/bin/env bash
if test x"${OS}" != x"FreeBSD"; then
source scl_source enable devtoolset-4
fi
FAKE_ROOT=/home/vagrant/fake.root
rm -rf ~/extract
@ -25,10 +29,6 @@ for pkg in /vagrant/pkgs/*.tar.gz; do
tar xfz $pkg
done
(cd autoconf-2.69 && ./configure --prefix=${FAKE_ROOT} && make && make install)
(cd automake-1.15 && ./configure --prefix=${FAKE_ROOT} && make && make install)
(cd make-4.2.1 && ./configure --prefix=${FAKE_ROOT} && make && make install)
OS=$(uname -s)
@ -57,27 +57,24 @@ if test x"${OS}" != x"FreeBSD"; then
--with-default-terminfo-dir=/usr/share/terminfo \
--enable-ext-colors \
--enable-widec \
CC="gcc44" \
CXX="g++44" \
&& \
make && make install)
(cd pcre-8.38 && \
(cd pcre-8.40 && \
./configure --prefix=${FAKE_ROOT} \
--enable-jit \
--enable-utf \
CC="gcc44" \
CXX="g++44" \
&& \
make && make install)
(cd zlib-1.2.8 && ./configure --prefix=${FAKE_ROOT} && make && make install)
(cd zlib-1.2.11 && ./configure --prefix=${FAKE_ROOT} && make && make install)
(cd libssh2-* &&
./configure --prefix=${FAKE_ROOT} \
--with-libssl-prefix=/home/vagrant/fake.root \
--with-libz-prefix=/home/vagrant/fake.root \
"LDFLAGS=-ldl" &&
"CPPFLAGS=-I${FAKE_ROOT}/include" \
"LDFLAGS=-ldl -L${FAKE_ROOT}/lib" &&
make &&
make install)
@ -100,14 +97,14 @@ else
&& \
make && make install)
(cd pcre-8.38 && \
(cd pcre-8.40 && \
./configure --prefix=${FAKE_ROOT} \
--enable-jit \
--enable-utf \
&& \
make && make install)
(cd zlib-1.2.8 && ./configure --prefix=${FAKE_ROOT} "CFLAGS=-fPIC" \
(cd zlib-1.2.11 && ./configure --prefix=${FAKE_ROOT} "CFLAGS=-fPIC" \
&& make && make install)
(cd libssh2-* &&

2
src/CMakeLists.txt
View File

@ -40,6 +40,7 @@ set(diag_STAT_SRCS
readline_curses.cc
readline_highlighters.cc
readline_possibilities.cc
regexp_vtab.cc
relative_time.cc
session_data.cc
sequence_matcher.cc
@ -115,6 +116,7 @@ set(diag_STAT_SRCS
pthreadpp.hh
readline_callbacks.hh
readline_possibilities.hh
regexp_vtab.hh
relative_time.hh
sequence_sink.hh
shlex.hh

2
src/Makefile.am
View File

@ -180,6 +180,7 @@ noinst_HEADERS = \
readline_curses.hh \
readline_highlighters.hh \
readline_possibilities.hh \
regexp_vtab.hh \
relative_time.hh \
sequence_matcher.hh \
sequence_sink.hh \
@ -274,6 +275,7 @@ libdiag_a_SOURCES = \
readline_curses.cc \
readline_highlighters.cc \
readline_possibilities.cc \
regexp_vtab.cc \
relative_time.cc \
session_data.cc \
sequence_matcher.cc \

2
src/extension-functions.c
View File

@ -120,6 +120,8 @@ Original code 2006 June 05 by relicoder.
#define SQLITE_SOUNDEX 1
#define HAVE_TRIM 1 /* LMH 2007-03-25 if sqlite has trim functions */
#define __STDC_FORMAT_MACROS
#ifdef COMPILE_SQLITE_EXTENSIONS_AS_LOADABLE_MODULE
#include "sqlite3ext.h"
SQLITE_EXTENSION_INIT1

2
src/file_vtab.cc
View File

@ -110,7 +110,7 @@ CREATE TABLE lnav_file (
int register_file_vtab(sqlite3 *db)
{
static vtab_module<tvt_no_update<lnav_file>> LNAV_FILE_MODULE;
static vtab_module<lnav_file> LNAV_FILE_MODULE;
int rc;

2
src/lnav.cc
View File

@ -120,6 +120,7 @@
#include "pretty_printer.hh"
#include "all_logs_vtab.hh"
#include "file_vtab.hh"
#include "regexp_vtab.hh"
#ifdef HAVE_LIBCURL
#include <curl/curl.h>
@ -2855,6 +2856,7 @@ int main(int argc, char *argv[])
register_environ_vtab(lnav_data.ld_db.in());
register_views_vtab(lnav_data.ld_db.in());
register_file_vtab(lnav_data.ld_db.in());
register_regexp_vtab(lnav_data.ld_db.in());
lnav_data.ld_vtab_manager =
new log_vtab_manager(lnav_data.ld_db,

2
src/log_vtab_impl.cc
View File

@ -652,7 +652,7 @@ static int vt_best_index(sqlite3_vtab *tab, sqlite3_index_info *p_info)
p_info->idxNum = argvInUse;
p_info->idxStr = (char *) index_copy;
p_info->needToFreeIdxStr = 1;
p_info->estimatedCost = 1.0;
p_info->estimatedCost = 10.0;
}
return SQLITE_OK;

2
src/pcrepp.hh
View File

@ -120,7 +120,7 @@ public:
/**
* @return a capture_t that covers all of the text that was matched.
*/
capture_t *all() { return pc_captures; };
capture_t *all() const { return pc_captures; };
/** @return An iterator to the first capture. */
iterator begin() { return pc_captures + 1; };

1
src/ptimec.hh
View File

@ -33,6 +33,7 @@
#define __pctimec_hh
// XXX
#define __STDC_FORMAT_MACROS
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

252
src/regexp_vtab.cc Normal file
View File

@ -0,0 +1,252 @@
/**
* Copyright (c) 2017, Timothy Stack
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice, this
* list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* * Neither the name of Timothy Stack nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ''AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
* ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include "config.h"
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include "lnav.hh"
#include "auto_mem.hh"
#include "lnav_log.hh"
#include "sql_util.hh"
#include "file_vtab.hh"
#include "vtab_module.hh"
using namespace std;
enum {
RC_COL_MATCH_INDEX,
RC_COL_INDEX,
RC_COL_NAME,
RC_COL_CAPTURE_COUNT,
RC_COL_RANGE_START,
RC_COL_RANGE_STOP,
RC_COL_CONTENT,
RC_COL_VALUE,
RC_COL_PATTERN,
};
struct regexp_capture {
using iterator = vector<logfile *>::iterator;
static constexpr const char *CREATE_STMT = R"(
CREATE TABLE regexp_capture (
match_index integer,
capture_index integer,
capture_name text,
capture_count integer,
range_start integer,
range_stop integer,
content text,
value text HIDDEN,
pattern text HIDDEN
);
)";
struct vtab {
sqlite3_vtab base;
operator sqlite3_vtab *() {
return &this->base;
};
};
struct cursor {
sqlite3_vtab_cursor base;
unique_ptr<pcrepp> c_pattern;
pcre_context_static<30> c_context;
unique_ptr<pcre_input> c_input;
string c_pattern_string;
string c_content;
int c_index;
int c_start_index;
bool c_matched;
int c_match_index;
sqlite3_int64 c_rowid;
cursor(sqlite3_vtab *vt)
: base({vt}),
c_index(0),
c_start_index(0),
c_match_index(0),
c_rowid(0) {
this->c_context.set_count(0);
};
int next() {
if (this->c_index >= (this->c_context.get_count() - 1)) {
this->c_input->pi_offset = this->c_input->pi_next_offset;
this->c_matched = this->c_pattern->match(this->c_context, *(this->c_input));
this->c_index = -1;
this->c_match_index += 1;
}
if (!this->c_pattern || !this->c_matched) {
return SQLITE_OK;
}
this->c_index += 1;
return SQLITE_OK;
};
int eof() {
return !this->c_pattern || !this->c_matched;
};
int get_rowid(sqlite3_int64 &rowid_out) {
rowid_out = this->c_rowid;
return SQLITE_OK;
};
};
int get_column(const cursor &vc, sqlite3_context *ctx, int col) {
pcre_context::capture_t &cap = vc.c_context.all()[vc.c_index];
switch (col) {
case RC_COL_MATCH_INDEX:
sqlite3_result_int64(ctx, vc.c_match_index);
break;
case RC_COL_INDEX:
sqlite3_result_int64(ctx, vc.c_index);
break;
case RC_COL_NAME:
if (vc.c_index == 0) {
sqlite3_result_null(ctx);
} else {
sqlite3_result_text(ctx, vc.c_pattern->name_for_capture(
vc.c_index - 1), -1, SQLITE_TRANSIENT);
}
break;
case RC_COL_CAPTURE_COUNT:
sqlite3_result_int64(ctx, vc.c_context.get_count());
break;
case RC_COL_RANGE_START:
sqlite3_result_int64(ctx, cap.c_begin);
break;
case RC_COL_RANGE_STOP:
sqlite3_result_int64(ctx, cap.c_end);
break;
case RC_COL_CONTENT:
if (cap.is_valid()) {
sqlite3_result_text(ctx,
vc.c_input->get_substr_start(&cap),
cap.length(),
SQLITE_TRANSIENT);
} else {
sqlite3_result_null(ctx);
}
break;
case RC_COL_VALUE:
sqlite3_result_text(ctx,
vc.c_content.c_str(),
vc.c_content.length(),
SQLITE_TRANSIENT);
break;
case RC_COL_PATTERN:
sqlite3_result_text(ctx,
vc.c_pattern_string.c_str(),
vc.c_pattern_string.length(),
SQLITE_TRANSIENT);
break;
}
return SQLITE_OK;
}
};
static int rcBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo)
{
vtab_index_constraints vic(pIdxInfo);
vtab_index_usage viu(pIdxInfo);
for (auto iter = vic.begin(); iter != vic.end(); ++iter) {
if (iter->op != SQLITE_INDEX_CONSTRAINT_EQ) {
continue;
}
switch (iter->iColumn) {
case RC_COL_VALUE:
case RC_COL_PATTERN:
viu.column_used(iter);
break;
}
}
viu.allocate_args(2);
return SQLITE_OK;
}
static int rcFilter(sqlite3_vtab_cursor *pVtabCursor,
int idxNum, const char *idxStr,
int argc, sqlite3_value **argv)
{
regexp_capture::cursor *pCur = (regexp_capture::cursor *)pVtabCursor;
if (argc != 2) {
pCur->c_content.clear();
pCur->c_pattern.reset();
return SQLITE_OK;
}
const char *value = (const char *) sqlite3_value_text(argv[0]);
const char *pattern = (const char *) sqlite3_value_text(argv[1]);
pCur->c_content = value;
pCur->c_pattern = make_unique<pcrepp>(pattern);
pCur->c_pattern_string = pattern;
pCur->c_index = 0;
pCur->c_context.set_count(0);
pCur->c_input = make_unique<pcre_input>(pCur->c_content);
pCur->c_matched = pCur->c_pattern->match(pCur->c_context, *(pCur->c_input));
return SQLITE_OK;
}
int register_regexp_vtab(sqlite3 *db)
{
static vtab_module<tvt_no_update<regexp_capture>> REGEXP_CAPTURE_MODULE;
int rc;
REGEXP_CAPTURE_MODULE.vm_module.xBestIndex = rcBestIndex;
REGEXP_CAPTURE_MODULE.vm_module.xFilter = rcFilter;
rc = REGEXP_CAPTURE_MODULE.create(db, "regexp_capture");
ensure(rc == SQLITE_OK);
return rc;
}

37
src/regexp_vtab.hh Normal file
View File

@ -0,0 +1,37 @@
/**
* Copyright (c) 2017, Timothy Stack
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice, this
* list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* * Neither the name of Timothy Stack nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ''AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
* ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __regexp_vtab_hh
#define __regexp_vtab_hh
#include <sqlite3.h>
int register_regexp_vtab(sqlite3 *db);
#endif

1
src/textview_curses.cc
View File

@ -217,7 +217,6 @@ void textview_curses::textview_value_for_row(vis_line_t row,
string_attrs_t & sa = value_out.get_attrs();
string & str = value_out.get_string();
highlight_map_t::iterator iter;
string::iterator str_iter;
text_format_t source_format = this->tc_sub_source->get_text_format();
this->tc_sub_source->text_value_for_line(*this, row, str);

114
src/vtab_module.hh
View File

@ -63,6 +63,107 @@ inline double from_sqlite<double>(sqlite3_value *val)
extern std::string vtab_module_schemas;
class vtab_index_constraints {
public:
vtab_index_constraints(const sqlite3_index_info *index_info)
: vic_index_info(*index_info) {
};
struct const_iterator {
const_iterator(vtab_index_constraints *parent, int index = 0)
: i_parent(parent), i_index(index) {
while (this->i_index < this->i_parent->vic_index_info.nConstraint &&
!this->i_parent->vic_index_info.aConstraint[this->i_index].usable) {
this->i_index += 1;
}
};
const_iterator& operator++() {
do {
this->i_index += 1;
} while (
this->i_index < this->i_parent->vic_index_info.nConstraint &&
!this->i_parent->vic_index_info.aConstraint[this->i_index].usable);
return *this;
};
const sqlite3_index_info::sqlite3_index_constraint &operator*() const {
return this->i_parent->vic_index_info.aConstraint[this->i_index];
};
const sqlite3_index_info::sqlite3_index_constraint *operator->() const {
return &this->i_parent->vic_index_info.aConstraint[this->i_index];
};
bool operator!=(const const_iterator &rhs) const {
return this->i_parent != rhs.i_parent || this->i_index != rhs.i_index;
};
const vtab_index_constraints *i_parent;
int i_index;
};
const_iterator begin() {
return const_iterator(this);
};
const_iterator end() {
return const_iterator(this, this->vic_index_info.nConstraint);
};
private:
const sqlite3_index_info &vic_index_info;
};
class vtab_index_usage {
public:
vtab_index_usage(sqlite3_index_info *index_info)
: viu_index_info(*index_info),
viu_used_column_count(0),
viu_max_column(0) {
};
void column_used(const vtab_index_constraints::const_iterator &iter) {
this->viu_max_column = std::max(iter->iColumn, this->viu_max_column);
this->viu_index_info.idxNum |= (1L << iter.i_index);
this->viu_used_column_count += 1;
};
void allocate_args(int expected) {
int n_arg = 0;
if (this->viu_used_column_count != expected) {
this->viu_index_info.estimatedCost = 2147483647;
this->viu_index_info.estimatedRows = 2147483647;
return;
}
for (int lpc = 0; lpc <= this->viu_max_column; lpc++) {
for (int cons_index = 0;
cons_index < this->viu_index_info.nConstraint;
cons_index++) {
if (this->viu_index_info.aConstraint[cons_index].iColumn != lpc) {
continue;
}
if (!(this->viu_index_info.idxNum & (1L << cons_index))) {
continue;
}
this->viu_index_info.aConstraintUsage[cons_index].argvIndex = ++n_arg;
}
}
this->viu_index_info.estimatedCost = 1.0;
this->viu_index_info.estimatedRows = 1;
};
private:
sqlite3_index_info &viu_index_info;
int viu_used_column_count;
int viu_max_column;
};
template<typename T>
struct vtab_module {
static int tvt_create(sqlite3 *db,
@ -196,6 +297,17 @@ struct vtab_module {
return vtab_module<T>::apply(handler, &T::update_row, tab, index, argc - 2, argv + 2);
};
template<typename U>
auto addUpdate(U u) -> decltype(&U::delete_row, void()) {
log_debug("has updates!");
this->vm_module.xUpdate = tvt_update;
};
template<typename U>
void addUpdate(...) {
log_debug("no has updates!");
};
vtab_module() noexcept {
memset(&this->vm_module, 0, sizeof(this->vm_module));
this->vm_module.iVersion = 0;
@ -211,7 +323,7 @@ struct vtab_module {
this->vm_module.xBestIndex = vt_best_index;
this->vm_module.xFilter = vt_filter;
this->vm_module.xColumn = tvt_column;
this->vm_module.xUpdate = tvt_update;
this->addUpdate<T>(T());
};
int create(sqlite3 *db, const char *name) {

3
test/drive_sql.cc
View File

@ -8,6 +8,7 @@
#include "lnav.hh"
#include "auto_mem.hh"
#include "sqlite-extension-func.h"
#include "regexp_vtab.hh"
struct callback_state {
int cs_row;
@ -65,6 +66,8 @@ int main(int argc, char *argv[])
register_collation_functions(db.in());
}
register_regexp_vtab(db.in());
if (sqlite3_exec(db.in(),
argv[1],
sql_callback,

15
test/test_sql.sh
View File

@ -497,7 +497,7 @@ EOF
schema_dump() {
${lnav_test} -n -c ';.schema' ${test_dir}/logfile_access_log.0 | head -n11
${lnav_test} -n -c ';.schema' ${test_dir}/logfile_access_log.0 | head -n12
}
run_test schema_dump
@ -508,6 +508,7 @@ CREATE VIRTUAL TABLE environ USING environ_vtab_impl();
CREATE VIRTUAL TABLE lnav_views USING lnav_views_impl();
CREATE VIRTUAL TABLE lnav_view_stack USING lnav_view_stack_impl();
CREATE VIRTUAL TABLE lnav_file USING lnav_file_impl();
CREATE VIRTUAL TABLE regexp_capture USING regexp_capture_impl();
CREATE TABLE http_status_codes (
status integer PRIMARY KEY,
message text,
@ -884,3 +885,15 @@ value
10
<NULL>
EOF
run_test ${lnav_test} -n \
-c ";SELECT regexp_capture.content FROM access_log, regexp_capture(access_log.cs_version, 'HTTP/(\d+\.\d+)') WHERE regexp_capture.capture_index = 1" \
-c ':write-csv-to -' \
${test_dir}/logfile_access_log.0
check_output "joining log table with regexp_capture is not working?" <<EOF
content
1.0
1.0
1.0
EOF

177
test/test_sql_str_func.sh
View File

@ -142,3 +142,180 @@ check_output "" <<EOF
Row 0:
Column result: {"foo":"abc","col_0":123.456}
EOF
run_test ./drive_sql "SELECT * FROM regexp_capture('foo bar', '\w+ (\w+)')"
check_output "" <<EOF
Row 0:
Column match_index: 0
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 2
Column range_start: 0
Column range_stop: 7
Column content: foo bar
Row 1:
Column match_index: 0
Column capture_index: 1
Column capture_name:
Column capture_count: 2
Column range_start: 4
Column range_stop: 7
Column content: bar
EOF
run_test ./drive_sql "SELECT * FROM regexp_capture('foo bar', '\w+ \w+')"
check_output "" <<EOF
Row 0:
Column match_index: 0
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 1
Column range_start: 0
Column range_stop: 7
Column content: foo bar
EOF
run_test ./drive_sql "SELECT * FROM regexp_capture('foo bar', '\w+ (?<word>\w+)')"
check_output "" <<EOF
Row 0:
Column match_index: 0
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 2
Column range_start: 0
Column range_stop: 7
Column content: foo bar
Row 1:
Column match_index: 0
Column capture_index: 1
Column capture_name: word
Column capture_count: 2
Column range_start: 4
Column range_stop: 7
Column content: bar
EOF
run_test ./drive_sql "SELECT * FROM regexp_capture('foo bar', '(bar)|\w+ (?<word>\w+)')"
check_output "" <<EOF
Row 0:
Column match_index: 0
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 3
Column range_start: 0
Column range_stop: 7
Column content: foo bar
Row 1:
Column match_index: 0
Column capture_index: 1
Column capture_name:
Column capture_count: 3
Column range_start: -1
Column range_stop: -1
Column content: (null)
Row 2:
Column match_index: 0
Column capture_index: 2
Column capture_name: word
Column capture_count: 3
Column range_start: 4
Column range_stop: 7
Column content: bar
EOF
run_test ./drive_sql "SELECT * FROM regexp_capture()"
check_output "" <<EOF
EOF
run_test ./drive_sql "SELECT * FROM regexp_capture('foo bar')"
check_output "" <<EOF
EOF
run_test ./drive_sql "SELECT * FROM regexp_capture('1 2 3 45', '(\d+)')"
check_output "" <<EOF
Row 0:
Column match_index: 0
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 2
Column range_start: 0
Column range_stop: 1
Column content: 1
Row 1:
Column match_index: 0
Column capture_index: 1
Column capture_name:
Column capture_count: 2
Column range_start: 0
Column range_stop: 1
Column content: 1
Row 2:
Column match_index: 1
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 2
Column range_start: 2
Column range_stop: 3
Column content: 2
Row 3:
Column match_index: 1
Column capture_index: 1
Column capture_name:
Column capture_count: 2
Column range_start: 2
Column range_stop: 3
Column content: 2
Row 4:
Column match_index: 2
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 2
Column range_start: 4
Column range_stop: 5
Column content: 3
Row 5:
Column match_index: 2
Column capture_index: 1
Column capture_name:
Column capture_count: 2
Column range_start: 4
Column range_stop: 5
Column content: 3
Row 6:
Column match_index: 3
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 2
Column range_start: 6
Column range_stop: 8
Column content: 45
Row 7:
Column match_index: 3
Column capture_index: 1
Column capture_name:
Column capture_count: 2
Column range_start: 6
Column range_stop: 8
Column content: 45
EOF
run_test ./drive_sql "SELECT * FROM regexp_capture('foo foo', '^foo')"
check_output "" <<EOF
Row 0:
Column match_index: 0
Column capture_index: 0
Column capture_name: (null)
Column capture_count: 1
Column range_start: 0
Column range_stop: 3
Column content: foo
EOF