Plugins

Introduction

The Falco libraries and Falco itself can be extended by using Plugins. Plugins are shared libraries that conform to a documented API and allow for:

• adding new event sources that can be evaluated using filtering expressions/Falco rules.
• adding the ability to define new fields that can extract information from events.

This page describes how plugins fit into the existing event processing pipeline and how to enable/configure plugins in Falco.

Plugins Architecture Concepts

Plugins

Plugins are dynamic shared libraries (.so files in Unix, .dll files in Windows) that export C calling convention functions. Programs like Falco dynamically load these libraries and call the exported functions to extend Falco's support for event sources/fields.

Plugins are versioned using semantic versioning to minimize regressions and compatibility issues.

Plugins can be written in any language, as long as they export the required functions. Go, however, is the preferred language to write plugins, followed by C/C++.

Plugins can implement one or more capabilities. In the scope of plugins, a capability is an extension of Falco's features in the form of a specific set of C function symbols exported by shared libraries. Currently, there are two plugin capabilities supported by the framework: event sourcing and field extraction.

Plugins are Coresident with Falco

The libraries will do everything possible to validate the data coming from the plugins and protect Falco and the other consumers from corrupted data. However, for performance reasons, plugins are "trusted": they run in the same thread and address space as Falco and they could crash the program. We assume that the user will be in control of plugin loading and will make sure only trusted plugins are loaded/packaged with Falco.

Plugin SDKs

To make it easier to write plugins, there are Go and C++ SDKs that handle the details of memory management and type conversion. These SDKs provide a streamlined way to implement plugins without having to deal with all the details of the lower-level functions that make up the Plugin API.

These SDKs are optional, but using them is highly recommended.

Event Sourcing Capability

Plugins with event sourcing capability provide a new event source and make it available to libscap and libsinsp. They have the ability to "open" and "close" a stream of events and return those events to the plugin framework. They also provide a plugin ID, which is globally unique and is used in capture files (see below). Event sources provided by plugins with this capability are tied to the events they generate and can be used by plugins with field extraction capabilities and within Falco rules (see below).

Field Extraction Capability

Plugins with field extraction capability have the ability to extract information from events based on fields. For example, a field (e.g. proc.name) extracts a value (e.g. process name like nginx) from a syscall event. The plugin returns a set of supported fields, and there are functions to extract a value given an event and field. The plugin framework can then build filtering expressions (e.g. rule conditions) based on these fields combined with relational and/or logical operators. For example, given the expression ct.name=root and ct.region=us-east-1, the plugin framework handles parsing the expression, calling the plugin to extract values for fields ct.name/ct.region for a given event, and determining the result of the expression. In a Falco output string like An EC2 Node was created (name=%ct.name region=%ct.region), the plugin framework handles parsing the output string, calling the plugin to extract values for fields, and building the resolved string, replacing the template field names (e.g. %ct.region) with values (e.g. us-east-1).

Plugins with this capability only focus on field extraction from events generated by other plugins or by the core libraries. They do not provide an event source but can extract fields from other event sources. The supported field extraction can be generic or be tied to a specific event source. An example is json field extraction, where a plugin might be able to extract fields from generic json payloads.

Composability of Capabilities

Plugin capabilities are composable, meaning that a single plugin can implement one or more capabilities. At loading time, the framework is able to recognize which capabilities the plugin correctly implements, and uses it accordingly inside Falco and the libraries. There can be plugins implementing event sourcing only, field extraction only, or both. For example, the AWS CloudTrail plugin implements both capabilities in order to provide the framework with the aws_cloudtrail event source and to allow extracting fields such as ct.name from the events it produces.

Plugin Event IDs

Every plugin with event sourcing capability requires its own, unique plugin event ID to interoperate with Falco and the other plugins. This ID is used in the following ways:

• The ID is saved in in-memory event objects and is used to identify the associated plugin that injected the event.
• The ID is saved in capture files and is used to recreate in-memory event objects when reading capture files.

The ID must be unique to ensure that events written by a given plugin will be properly associated with that plugin (and its event sources, see below).

Plugin authors must register the plugin with the Falcosecurity organization by creating a PR to modify the plugin registry with details on the new plugin. This ensures that a given ID is used by exactly one plugin.

Plugin Event Sources and Interoperability

Events returned by plugins with event sourcing capability have an event source that describes the event's information. This is distinct from the plugin name to allow for multiple plugin implementations to generate the same kind of events. For example, there might be plugins gke-k8saudit, eks-k8saudit, ibmcloud-k8saudit, etc. that all fetch K8s Audit information. The plugins would have different names and IDs but would have the same event source k8s_audit.

A plugin with field extraction capability optionally provides a set of compatible event sources. When the framework receives an event with an event source in the plugin's set of event sources, fields in expressions (i.e., fields included in the rule's output field) will be extracted from events using the plugin. The set of compatible event sources can also be omitted. In this case, all events will be presented to the plugin, regardless of their source. In this case, the plugin must detect the format of arbitrary payloads and be able to return NULL/no value when the payload is not supported. As such, given a specific event source such as k8s_audit, there is an implicit contract to honor regarding how data is formatted in each event of that source, such that compatible plugins with field extraction capability are able to parse events of a certain source even if they are produced by different plugins.

Plugin authors should register the plugin with the Falcosecurity organization by creating a PR to modify the plugin registry file with details on the new plugin. This allows plugin authors to coordinate about event source data formats.

Handling Duplicate/Overlapping Fields in Plugins/Libraries Core

At an initial glance, adding plugins introduces the possibility of tens/hundreds of new filtercheck fields that could potentially overlap/conflict. For example, what happens if a plugin defines a proc.name field? However, the notion of event source makes these potential conflicts manageable.

Remember that field extraction is always done in the context of an event, and each event can be mapped back to an event source. So we only need to ensure that filtercheck fields are non-overlapping for a given event source. For example, it's perfectly valid for an AWS CloudTrail plugin to define a proc.name field, as the events generated by that plugin are wholly separate from syscall events. For syscall events, the AWS CloudTrail plugin is not involved and the core libraries extract the process name for the tid performing a syscall. For AWS CloudTrail events, the core libraries are not involved in field extraction. Extraction is performed by the AWS CloudTrail plugin instead.

When managing plugins, we only need to ensure the following:

• That only one plugin is loaded at a time that exports a given event source. For example, the libraries can load either a gke-k8saudit plugin with event source k8s_audit, or eks-k8saudit with event source k8s_audit, but not both.
• That for a mix of plugins with event sourcing and field extraction capabilities having the same event source, that the fields are distinct. For example, a plugin event sourcing capability providing the source k8s_audit can export ka.* fields, and a plugin with field extractor capabilities with event source k8s_audit can export a jevt.value[/...] field, and the appropriate plugin will be used to extract fields from k8s_audit events as fields are parsed from condition expressions/output format strings.

Plugin API

Here is an overview of the functions that comprise the plugin API. This list is not extensive: the plugin API reference has full documentation of plugin APIs for all the capabilities supported by the framework.

In almost all cases, a plugin author can use the SDKs which provide a more streamlined interface. This still provides a good overview of the functionality a plugin provides.

Info Functions

A set of functions provide information about the plugin and its compatibility with the plugin framework:

• plugin_get_required_api_version: Return the version of the plugin API used by this plugin.
• plugin_get_id: Return the unique ID of the plugin (event sourcing capability only).
• plugin_get_name: Return the name of the plugin.
• plugin_get_description: Return a short description of the plugin.
• plugin_get_contact: Return a contact url/email/twitter account for the plugin authors.
• plugin_get_version: Return the version of the plugin itself.
• plugin_get_event_source: Return a string describing the events generated by this plugin (event sourcing capability only).
• plugin_get_init_schema: (Optional) Return a string describing a schema for the configuration passed to plugin_init.
• plugin_list_open_params: (Optional) Return a list of suggested valid parameter values for plugin_open.
• plugin_get_extract_event_sources: (Optional) Return a list of all compatible event sources (field extraction capability only).

Instance/Capture Management Functions

Plugins have functions to initialize/destroy a plugin, as well as functions to open/close streams of events:

• plugin_init: Initialize the plugin and, if needed, allocate its state.
• plugin_destroy: Destroy the plugin and, if plugin state was allocated, free it.
• plugin_open: Open the source and start a stream of events (event sourcing capability only).
• plugin_close: Close a stream of events (event sourcing capability only).

Events/Fields Related Functions

Plugins with event sourcing capability have functions to provide events to the plugin framework:

• plugin_next_batch: Return one or more events to the plugin framework.
• plugin_get_progress: (Optional) Provide feedback on how much of the event stream has been read.
• plugin_event_to_string: (Optional) Return a text representation of an event generated by this plugin.

Plugins with field extraction capability have functions to define the set of fields that can be used to extract information from events, to actually extract values from events, and to return printable representations of events:

• plugin_get_fields: Return the list of extractor fields exported by this plugin.
• plugin_extract_fields: Extract one or more a filter field values from an event.

How Falco Uses Plugins

Falco loads plugins based on configuration in falco.yaml. Currently, if a plugin with event sourcing capability is loaded then the only events processed are from that plugin; syscall events are disabled. There are other restrictions on loaded plugins (see below).

Loading plugins in Falco

Falco configures plugins via the new "plugins" property in falco.yaml. Here's an example:

plugins:
  - name: cloudtrail
    library_path: libcloudtrail.so
    init_config: ""
    open_params: ""
  - name: json
    library_path: libjson.so
    init_config: ""

# Optional
load_plugins: [cloudtrail, json]

A new plugins property in falco.yaml will define the set of plugins that can be loaded by Falco, and a new load_plugins property will control which plugins are actually loaded when Falco starts.

For more information, see Configuration.

The mechanics of loading a plugin are implemented in the libraries and leverage the dynamic library functionality of the operating system (dlopen/dlsym in unix, LoadLibrary/GetProcAddress in Windows). The plugin loading code also ensures that:

• the plugin is valid, i.e. that it exports the set of expected symbols
• the plugin has an api version number that is compatible with the plugin framework.
• that only one plugin with event sourcing capability is loaded at a time for a given event source
• if a mix of plugins for both event sourcing and field extraction are loaded for a given event source, that the exported fields have unique names that don't overlap across plugins

Event Sources and Falco Rules

Falco rules already have the notion of a source, using the source property in YAML rules objects, and there is primarily one kind of event source: syscall. The source property in Falco rules maps a given rule to the event source on which the rule runs.

For example, given a plugin providing events with source aws_cloudtrail, and a Falco rule with source property aws_cloudtrail, the rule will be evaluated for any events returned by the AWS CloudTrail plugin.

Similarly, a plugin with field extraction capability that includes aws_cloudtrail in its set of event sources will have the opportunity to extract information from CloudTrail events. As a result, fields exported by the plugin can be put in a rule's condition, exception, or output properties when the rule has a source aws_cloudtrail.

Falco compiles rules/macros/lists selectively based on the set of loaded plugins (specifically, their event sources), instead of unconditionally as Falco is started. This is especially important for macros, which do not contain a source property, but might contain fields that are only implemented by a given plugin.

Plugin Versions and Falco Rules

To allow rules files to document the plugin versions they are compatible with, rules files can have a new top-level field required_plugin_versions. The field is optional, and if not provided no plugin compatibility checks will be performed. The syntax of required_plugin_versions is the following:

- required_plugin_versions:
  - name: <plugin_name>
    version: x.y.z
  ...

Below required_plugin_versions is a list of objects, where each object has name and version properties. If a plugin is loaded, and if an entry in required_plugin_versions has a matching name, then the loaded plugin version must be semver compatible with the version property.

Falco can load multiple rules files, and each file may contain its own required_plugin_versions property. In this case, name+version pairs across all files will be merged, and in the case of duplicate names all provided versions must be compatible.

Plugin Developer's Guide

If you are interested in authoring your own plugin, or modifying an existing plugin to add new functionality, we've written a developer's guide that documents the full plugin APIs and walks through two existing plugins to show how the API is used.

Where's the Code?

Plugins

Plugins hosted and maintained by the Falcosecurity community are at the plugins GitHub repository. There, you can also find the plugin registry containing info about all the plugins officially recognized by the Falcosecurity organization, which include both the ones hosted by the community and the external ones.

Plugin SDKs

To facilitate the development of plugins written in Go, we've written a SDK that provides support code for writing plugins. The SDK provides Go structs/enums corresponding to the C structs/enums used by the API, has utility packages that manage the details of memory management/type conversion, and presents abstract interfaces that provide a more streamlined interface to potential plugin authors. We go through the details and the architecture of the GO SDK in the Go SDK walkthrough section.

There is also an experimental C++ SDK that defines abstract C++ base classes for plugins. Plugin authors can derive from these base classes and implement the abstract methods to provide demographic information, events, and extract fields from events. This SDK is currently in line with the most recent changes in the plugin API, and will be subject to many updates in the near future.

All of the Falcosecurity-provided plugins use these SDKs.

Falco

Falco itself includes the k8saudit, cloudtrail and json plugins in its packages and container images. The plugins are defined in falco.yaml but by default, no plugins are loaded when Falco starts.

To add plugins, you can put them as shared libraries below /usr/share/falco/plugins, and use a relative path in the value for library_path in falco.yaml.

References

If you're interested in how this feature came about, you can view the original proposal for the plugin system.