Text line filter¶
Metadata¶
- name:
saturnin.text.linefilter
- description:
Text line filter microservice
- classification:
text/filter
- OID:
1.3.6.1.4.1.53446.1.1.0.3.1.3
- OID name:
iso.org.dod.internet.private.enterprise.firebird.butler.platform.saturnin.micro.text.linefilter
- UUID:
455093ea-fd99-53c4-9b1f-b2c9b764f482
- facilities:
None
- API:
None
Usage¶
This microservice is a DATA FILTER that uses FBDP protocol to read blocks of text from input data pipe, and write lines that meet the specified conditions as blocks of text into output data pipe.
A filter can be defined as one of the following:
regex expression
Python bool expression that uses
line
to refer to the line of text being processedPython function with signature
def fname(line: str) -> bool
Configuration¶
- agent:
UUID
: Agent identification (service UUID)- logging_id:
str
: Logging ID for this component instance, see Context-based logging for details.- propagate_input_error:
bool
: When input pipe is closed with error, close output with error as well. DEFAULTTrue
.- input_pipe:
str
: Input Data Pipe Identification. REQUIRED option.- input_pipe_address:
ZMQAddress
: Input Data Pipe endpoint address. REQUIRED option.- input_pipe_mode:
SocketMode
: Input Data Pipe Mode (bind/connect). REQUIRED option.- input_pipe_format:
MIME
: Input Pipe data format specification. REQUIRED for CONNECT pipe mode.- input_batch_size:
int
: Input Pipe Data batch size. DEFAULT 50.- input_ready_schedule_interval:
int
: Input Pipe READY message schedule interval in milliseconds. See FBDP documentation for details. DEFAULT 1000.- output_pipe:
str
: Output Data Pipe Identification. REQUIRED option.- output_pipe_address:
ZMQAddress
: Output Data Pipe endpoint address. REQUIRED option.- output_pipe_mode:
SocketMode
: Output Data Pipe Mode (bind/connect). REQUIRED option.- output_pipe_format:
MIME
: Output Pipe data format specification. REQUIRED for CONNECT pipe mode.- output_batch_size:
int
: Output Pipe Data batch size. DEFAULT 50.- output_ready_schedule_interval:
int
: Output Pipe READY message schedule interval in milliseconds. See FBDP documentation for details. DEFAULT 1000.- max_chars:
int
: Max. number of characters transmitted in one message. DEFAULT 65535.- regex:
str
: Filter by regular expression- expr:
PyExpr
: Filter by Python expression (useline
to refer to the line of text being processed)- func:
PyCallable
: Filter by Python function. Python function with signature:def fname(line: str) -> bool
Important
Exactly one filter definition must be provided.
Only ‘text/plain’ MIME types are alowed for input and output
pipe_format
.Only ‘charset’ and ‘errors’ MIME parameters are alowed for input and output
pipe_format
.Regex must be valid.