mutate

The mutate filter allows you to perform general mutations on fields. You can rename, remove, replace, and modify fields in your events.

TODO(sissel): Support regexp replacements like String#gsub ?

 

Synopsis

This plugin supports the following configuration options:

Required configuration options:

mutate {
}

Available configuration options:

Details

 

add_field

  • Value type is hash
  • Default value is {}

If this filter is successful, add any arbitrary fields to this event. Field names can be dynamic and include parts of the event using the %{field}.

Example:

filter {
  mutate {
    add_field => { "foo_%{somefield}" => "Hello world, from %{host}" }
  }
}
# You can also add multiple fields at once:
filter {
  mutate {
    add_field => {
      "foo_%{somefield}" => "Hello world, from %{host}"
      "new_field" => "new_static_value"
    }
  }
}

If the event has field "somefield" == "hello" this filter, on success, would add field foo_hello if it is present, with the value above and the %{host} piece replaced with that value from the event. The second example would also add a hardcoded field.

add_tag

  • Value type is array
  • Default value is []

If this filter is successful, add arbitrary tags to the event. Tags can be dynamic and include parts of the event using the %{field} syntax.

Example:

filter {
  mutate {
    add_tag => [ "foo_%{somefield}" ]
  }
}
# You can also add multiple tags at once:
filter {
  mutate {
    add_tag => [ "foo_%{somefield}", "taggedy_tag"]
  }
}

If the event has field "somefield" == "hello" this filter, on success, would add a tag foo_hello (and the second example would of course add a taggedy_tag tag).

convert

  • Value type is hash
  • There is no default value for this setting.

Convert a field’s value to a different type, like turning a string to an integer. If the field value is an array, all members will be converted. If the field is a hash, no action will be taken.

Valid conversion targets are: integer, float, string.

Example:

filter {
  mutate {
    convert => { "fieldname" => "integer" }
  }
}

exclude_tags (DEPRECATED)

  • DEPRECATED WARNING: This configuration item is deprecated and may not be available in future versions.
  • Value type is array
  • Default value is []

Only handle events without all/any (controlled by exclude_any config option) of these tags. Optional.

gsub

  • Value type is array
  • There is no default value for this setting.

Convert a string field by applying a regular expression and a replacement. If the field is not a string, no action will be taken.

This configuration takes an array consisting of 3 elements per field/substitution.

Be aware of escaping any backslash in the config file.

Example:

filter {
  mutate {
    gsub => [
      # replace all forward slashes with underscore
      "fieldname", "/", "_",
      # replace backslashes, question marks, hashes, and minuses with
      # dot
      "fieldname2", "[\\?#-]", "."
    ]
  }
}

join

  • Value type is hash
  • There is no default value for this setting.

Join an array with a separator character. Does nothing on non-array fields.

Example:

filter {
  mutate {
    join => { "fieldname" => "," }
  }
}

lowercase

  • Value type is array
  • There is no default value for this setting.

Convert a string to its lowercase equivalent.

Example:

filter {
  mutate {
    lowercase => [ "fieldname" ]
  }
}

merge

  • Value type is hash
  • There is no default value for this setting.

Merge two fields of arrays or hashes. String fields will be automatically be converted into an array, so:

`array` + `string` will work
`string` + `string` will result in an 2 entry array in `dest_field`
`array` and `hash` will not work

Example:

filter {
  mutate {
     merge => { "dest_field" => "added_field" }
  }
}

periodic_flush

  • Value type is boolean
  • Default value is false

Call the filter flush method at regular interval. Optional.

remove (DEPRECATED)

  • DEPRECATED WARNING: This configuration item is deprecated and may not be available in future versions.
  • Value type is array
  • There is no default value for this setting.

Remove one or more fields.

Example:

filter {
  mutate {
    remove => [ "client" ]  # Removes the 'client' field
  }
}

This option is deprecated, instead use remove_field option available in all filters.

remove_field

  • Value type is array
  • Default value is []

If this filter is successful, remove arbitrary fields from this event. Example:

filter {
  mutate {
    remove_field => [ "foo_%{somefield}" ]
  }
}
# You can also remove multiple fields at once:
filter {
  mutate {
    remove_field => [ "foo_%{somefield}", "my_extraneous_field" ]
  }
}

If the event has field "somefield" == "hello" this filter, on success, would remove the field with name foo_hello if it is present. The second example would remove an additional, non-dynamic field.

remove_tag

  • Value type is array
  • Default value is []

If this filter is successful, remove arbitrary tags from the event. Tags can be dynamic and include parts of the event using the %{field} syntax.

Example:

filter {
  mutate {
    remove_tag => [ "foo_%{somefield}" ]
  }
}
# You can also remove multiple tags at once:
filter {
  mutate {
    remove_tag => [ "foo_%{somefield}", "sad_unwanted_tag"]
  }
}

If the event has field "somefield" == "hello" this filter, on success, would remove the tag foo_hello if it is present. The second example would remove a sad, unwanted tag as well.

rename

  • Value type is hash
  • There is no default value for this setting.

Rename one or more fields.

Example:

filter {
  mutate {
    # Renames the 'HOSTORIP' field to 'client_ip'
    rename => { "HOSTORIP" => "client_ip" }
  }
}

replace

  • Value type is hash
  • There is no default value for this setting.

to help you build a new value from other parts of the event.

Example:

filter {
  mutate {
    replace => { "message" => "%{source_host}: My new message" }
  }
}

split

  • Value type is hash
  • There is no default value for this setting.

Split a field to an array using a separator character. Only works on string fields.

Example:

filter {
  mutate {
     split => { "fieldname" => "," }
  }
}

strip

  • Value type is array
  • There is no default value for this setting.

Strip whitespace from field. NOTE: this only works on leading and trailing whitespace.

Example:

filter {
  mutate {
     strip => ["field1", "field2"]
  }
}

tags (DEPRECATED)

  • DEPRECATED WARNING: This configuration item is deprecated and may not be available in future versions.
  • Value type is array
  • Default value is []

Only handle events with all/any (controlled by include_any config option) of these tags. Optional.

type (DEPRECATED)

  • DEPRECATED WARNING: This configuration item is deprecated and may not be available in future versions.
  • Value type is string
  • Default value is ""

Note that all of the specified routing options (type,tags,exclude_tags,include_fields, exclude_fields) must be met in order for the event to be handled by the filter. The type to act on. If a type is given, then this filter will only act on messages with the same type. See any input plugin’s "type" attribute for more. Optional.

update

  • Value type is hash
  • There is no default value for this setting.

Update an existing field with a new value. If the field does not exist, then no action will be taken.

Example:

filter {
  mutate {
    update => { "sample" => "My new message" }
  }
}

uppercase

  • Value type is array
  • There is no default value for this setting.

Convert a string to its uppercase equivalent.

Example:

filter {
  mutate {
    uppercase => [ "fieldname" ]
  }
}