Logstash, clone filter & add_field mysteries

That’s a really great piece of documentation. This does not work:

# let's clone each event, one goes to somewhere, one goes to somewhere else.
# note this was copy-pasted from the docs!
# see here: https://is.gd/QSHNps
# again. THIS DOES NOT WORK.
filter {
  clone {
    add_field => { "token" => "ABCDEF" }
  }
}
output {
  if [token] {
    # go somewhere
    tcp { ... }
  } else {
    # go to somewhere else
    s3 { ... }
  }
}

Why? Because the clone filter will not clone anything. And the documentation is super unclear on this. If you know it, you can read it - if you don’t know this, you’ll … google.

For it to actually clone anything you have to specify the ‘clones => [“one”, …]’ parameter. Then it will clone, and add the token field as expected. Like this:

filter {
  clone {
    clones => ["logz"]     # NOW it will clone.
    add_field => { "token" => "ABCDEF" }
  }
}

Interestingly the “clones =>” parameter is optional, which just confuses the shit out of me.

The reasoning that I don’t just add the field altogether is that this is the access token for our externally hosted ELK service. This should only be there for the external path, and not be put in S3 in parallel.

References:  

• https://logstash.jira.com/browse/LOGSTASH-2144