Text line filter

Metadata

name:

saturnin.text.linefilter

description:

Text line filter microservice

classification:

text/filter

OID:

1.3.6.1.4.1.53446.1.1.0.3.1.3

OID name:

iso.org.dod.internet.private.enterprise.firebird.butler.platform.saturnin.micro.text.linefilter

UUID:

455093ea-fd99-53c4-9b1f-b2c9b764f482

facilities:

None

API:

None

Usage

This microservice is a DATA FILTER that uses FBDP protocol to read blocks of text from input data pipe, and write lines that meet the specified conditions as blocks of text into output data pipe.

A filter can be defined as one of the following:

  • regex expression

  • Python bool expression that uses line to refer to the line of text being processed

  • Python function with signature def fname(line: str) -> bool

Configuration

agent:

UUID: Agent identification (service UUID)

logging_id:

str: Logging ID for this component instance, see Context-based logging for details.

propagate_input_error:

bool: When input pipe is closed with error, close output with error as well. DEFAULT True.

input_pipe:

str: Input Data Pipe Identification. REQUIRED option.

input_pipe_address:

ZMQAddress: Input Data Pipe endpoint address. REQUIRED option.

input_pipe_mode:

SocketMode: Input Data Pipe Mode (bind/connect). REQUIRED option.

input_pipe_format:

MIME: Input Pipe data format specification. REQUIRED for CONNECT pipe mode.

input_batch_size:

int: Input Pipe Data batch size. DEFAULT 50.

input_ready_schedule_interval:

int: Input Pipe READY message schedule interval in milliseconds. See FBDP documentation for details. DEFAULT 1000.

output_pipe:

str: Output Data Pipe Identification. REQUIRED option.

output_pipe_address:

ZMQAddress: Output Data Pipe endpoint address. REQUIRED option.

output_pipe_mode:

SocketMode: Output Data Pipe Mode (bind/connect). REQUIRED option.

output_pipe_format:

MIME: Output Pipe data format specification. REQUIRED for CONNECT pipe mode.

output_batch_size:

int: Output Pipe Data batch size. DEFAULT 50.

output_ready_schedule_interval:

int: Output Pipe READY message schedule interval in milliseconds. See FBDP documentation for details. DEFAULT 1000.

max_chars:

int: Max. number of characters transmitted in one message. DEFAULT 65535.

regex:

str: Filter by regular expression

expr:

PyExpr: Filter by Python expression (use line to refer to the line of text being processed)

func:

PyCallable: Filter by Python function. Python function with signature: def fname(line: str) -> bool

Important

  • Exactly one filter definition must be provided.

  • Only ‘text/plain’ MIME types are alowed for input and output pipe_format.

  • Only ‘charset’ and ‘errors’ MIME parameters are alowed for input and output pipe_format.

  • Regex must be valid.