Skip to content

Filterx borrow string slices #899

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
bazsi wants to merge 30 commits into
base: main
Choose a base branch
from
Open

Filterx borrow string slices #899

bazsi wants to merge 30 commits into from

Conversation

@bazsi
Copy link
Member

@bazsi bazsi commented Jan 6, 2026

This comes on top of #898 and allows us to not copy the content of string objects, only reference them by start/end offsets.

@bazsi bazsi force-pushed the filterx-borrow-string-slices branch from 560a297 to 730844b Compare January 7, 2026 08:49
@OverOrion OverOrion self-requested a review January 27, 2026 10:30
bazsi added 17 commits January 30, 2026 09:06
@bazsi
light: allow enough timeout for valgrind to start/stop syslog-ng
5bbe7a0 
Also reduce polling frequency as checking if a process exited every 1msec
is heating up my CPU.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
logmsg/logmsg: register missing predefined tag "syslog.missing_message"
7763428 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
str-utils: add strn_to_strz to compare str+len against a nul terminat…
4b296bc 
…ed string

Simply using strncmp() does not work, as the length needs to be checked too.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
str-utils: add strn_hash() function
3ad62da 
This is the same as g_str_hash() but with an explicit length argument.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
str-utils: add strn_eq_strn() function
3c86694 
This compares two, potentially not NUL terminated strings.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/filterx-plist: add support for list of FilterXObjects
ab5b9e2 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/object-string: add new value accessor functions
a626bab 
To match how callers intend to use the extracted fields, add a few more
variants of filterx_string_get_value_ref()

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/object-extractor: add similar accessors for strings as for Fi…
7dc86da 
…lterXString

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/object-message-value: remove assertion from potential fastpath
0315478 
We will crash in the next line if the length pointer is NULL anyway.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/expr-function: add strdup() versions of literal argument hand…
b111cac 
…ling functions

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/tests: use string+length string accessors
0b3d666 
Handle cases where FilterXString instances are not NUL terminated.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx: trivial fixes to handle non-NUL terminated strings
cc20cec 
By using the new accessor functions, using non-NUL terminated strings
became easier. This patch only contains the trivial changes.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/expr-regexp-common: add pattern_len argument to specify patte…
ae3256e 
…rn length

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/func-set-fields: fix len handling in key equal function
3c938db 
To work with strings that are not NUL terminated.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/func-str: remove _format_str_obj() wrapper
0edeea7 
This just calls filterx_object_extract_string_ref(), which we already use
in the code otherwise.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/func-flatten: properly handle non-nul terminated strings
48b174d 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/func-str-transform: fix missing NUL handling
9d45601 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi bazsi force-pushed the filterx-borrow-string-slices branch from 730844b to 47f233d Compare January 30, 2026 08:11
bazsi added 10 commits January 30, 2026 09:50
@bazsi
filterx/func-unset-empties: fix NUL termination issues & use FilterXO…
ba8209f 
…bjectList

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/object-string: handle non-NUL terminated strings in string dedup
693b1cc 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
cef/filterx: fix non-NUL terminated string issues
cfd95e2 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
xml/filterx: fix non-NUL terminated string issues
6f1bf9a 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
metrics: store explicit "length" fields for StatsClusterLabel
47288a0 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/object-string: use non-NUL terminator for FilterXString->str …
ce18e06 
…in DEBUG mode

This is to test if users of FilterXString properly handle the
`length` member or rely on the NUL terminator.

In production builds, this is not done, so in cases where we
do the NUL character, we don't need to use bounce buffers.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/filterx-object: prefer first match in filterx_object_is_type()
3edbe9e 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/object-string: add borrowed-slice constructor
b2881ec 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
csv-scanner: add csv_scanner_get_value_slice() implementation
b5f7e4a 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
csv/filterx: initialize value as a slice if possible
a179f3d 
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/object-message-value: create borrowed slices when unmarshalli…
35c378b 
…ng LM_VT_STRING

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi bazsi force-pushed the filterx-borrow-string-slices branch from 47f233d to 35c378b Compare January 30, 2026 08:51
bazsi added 2 commits February 8, 2026 10:56
@bazsi
filterx/object-string: extract _filterx_string_resolve_from_cache() f…
45ad73b 
…unction

So that it can be reused in the string slice path.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi
filterx/object-string: use cached string values for the string slice …
4cc9dd5 
…constructor

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@OverOrion OverOrion Awaiting requested review from OverOrion

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bazsi