apache log x fluentd x datadog

Based on this article , we can setup apache log processor to forward data to datadog.

Mind the format setting, change ltsv => apache2

<source>
    type tail
    format apache2
    #time_format %d/%b/%Y:%H:%M:%S %z
    path /var/log/httpd/access_log
    pos_file /tmp/access_log.pos
    tag apache.access
</source>

Required plugins:

fluent-plugin-datacounter

fluent-plugin-rename-key

fluent-plugin-record-reformer

fluent-plugin-forest

fluent-plugin-dd

If you are using td-agent, plugin can be installed using: sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-record-reformer
Permission issue may arise, so:chmod 0645 /var/log/httpd
Shoot, apparently we need to tweak to http status matching as well. Based on this article , we should math the code instead of status key

<match apache.access>
    type datacounter
    tag apache.status
    aggregate tag
    count_key status
    pattern1 2xx ^2\d\d$
    pattern2 3xx ^3\d\d$
    pattern3 4xx ^4\d\d$
    pattern4 5xx ^5\d\d$
</match>

Voila



Now we're talking ;)

PS:
Both fluentd and datadog are new to me