ModSecurity: Logging and debugging

ModSecurity: Logging and debugging

table.nginx-blog, table.nginx-blog th, table.nginx-blog td {
border: 2px solid black;
border-collapse: collapse;
}
table.nginx-blog {
width: 100%;
}
table.nginx-blog th {
background-color: #d3d3d3;
align: left;
padding-left: 5px;
padding-right: 5px;
padding-bottom: 2px;
padding-top: 2px;
line-height: 120%;
}
table.nginx-blog td {
padding-left: 5px;
padding-right: 5px;
padding-bottom: 2px;
padding-top: 5px;
line-height: 120%;
}
table.nginx-blog td.center {
text-align: center;
padding-bottom: 2px;
padding-top: 5px;
line-height: 120%;
}

When something is not working as you expect it to, logs are always the first place to look. Good logs can provide valuable insights to help you troubleshoot the problems you’re facing. One of the reasons Ivan Ristic originally created ModSecurity is that he was frustrated with the lack of visibility in the tools he was using. It’s no surprise, then, that ModSecurity has extensive logging and debugging capabilities.

ModSecurity has two types of logs:

• An audit log. For every transaction that’s blocked, ModSecurity provides detailed logs about the transaction and why it was blocked.
• A debug log. When turned on, this log keeps extensive information about everything that ModSecurity does.

The audit log is useful for learning not just why an individual attack was blocked, but for finding out more about overall attack patterns. You might be surprised by how much bot and scanner traffic you get just by exposing ports 80 and/or 443 to the Internet.

In this blog post, we’ll describe the basics of logging and debugging with ModSecurity.

Audit log

The main log in ModSecurity is the audit log, which logs all attacks, including potential attacks, that occur. If you’ve followed our installation instructions for ModSecurity (OSS) or the NGINX WAF (Plus), then by default, ModSecurity will log all transactions that triggered a warning or error, as well as all transactions that resulted in 5xx and 4xx responses, except for 404. (For an Ubuntu 16.04 system only, the audit log is in /var/log/modsec_audit.log.)

The ModSecurity audit log is partitioned into sections. This makes it easier to scan the log and find the information you’re looking for. The table below outlines what each section contains:

Each transaction that triggers an audit log entry will have any or all of the above sections logged. You can configure which sections are logged.

Audit log example

A sample ModSecurity audit log entry might look like this:

---ICmPEb5c---A--

[04/Oct/2017:21:45:15 +0000] 150715351558.929952 141.212.122.16 64384 141.212.122.16 80

---ICmPEb5c---B--

GET / HTTP/1.1

Host: 54.183.57.254

User-Agent: Mozilla/5.0 zgrab/0.x

Accept-Encoding: gzip
---ICmPEb5c---D--
---ICmPEb5c---F--

HTTP/1.1 200

Server: nginx/1.13.4

Date: Wed, 04 Oct 2017 21:45:15 GMT

Content-Type: text/html

Connection: keep-alive
---ICmPEb5c---H--

ModSecurity: Warning. Matched "Operator `Rx' with parameter `^[d.:]+$' against variable `REQUEST_HEADERS:Host' (Value: `54.183.57.254' ) [file "/root/owasp

-v3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "733"] [id "920350"] [rev "2"] [msg "Host header is a numeric IP address"] [data "54.183.57.254"] [s

everity "4"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "9"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-prot

ocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/IP_HOST"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [ref "o0,13v21,13"]
---ICmPEb5c---I--
---ICmPEb5c---J--
---ICmPEb5c---Z--

Though it’s not immediately apparent from the table above, the best section to find information on why a particular request was blocked is section H, not section K. From the above audit log example, if we scan through section H, we can see the message "Host header is a numeric IP address" which indicates someone tried to access our site by IP address rather than by hostname. This may be indicative of a scanner.

Audit logging configuration

If you followed our instructions for installing and configuring ModSecurity, you’ll find the audit logging configuration in /etc/nginx/modsec/modsecurity.conf. In that file, you’ll see the following three directives that control what is put into the audit log:

SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABIJDEFHZ

• SecAuditEngine — Controls whats should be logged. Options are:
  Off — Disable the audit log. On — Log all transactions, which can can useful when debugging. RelevantOnly — Only log transactions that have triggered a warning/error, or have a status code that matches what’s in the SecAuditLogRelevantStatus directive.
• SecAuditLogRelevantStatus — If SecAuditEngine is set to RelevantOnly, then this directive controls what HTTP response status codes should be logged. It’s regular expression-based. The above value will log all 5xx and 4xx responses, excluding 404s.
• SecAuditLogParts — Controls what sections should be included in the access log. Removing sections you’re not interested in will reduce the size of the audit log and make it easier to scan.

Note: For additional audit-logging configuration directives, please refer to the ModSecurity wiki.

Debug log

When the debug log is turned on, it provides a wealth of information on everything ModSecurity does. For troubleshooting issues as to why something is not working the way you expect it to, the debug log is your go-to resource. It’s also great if you’re getting started with ModSecurity and want to observe why it does things a certain way.

Debug log example

The debug log will look like the below. It has a lot of details on the actions ModSecurity takes for any and all transactions:

[4] (Rule: 1234) Executing operator "Contains" with param "test" against ARGS:testparam.

[9] Target value: "test" (Variable: ARGS:testparam)

[9] Matched vars updated.

[4] Running [independent] (non-disruptive) action: log

[9] Saving transaction to logs

[4] Rule returned 1.

[9] (SecDefaultAction) Running action: log

[9] Saving transaction to logs

[9] (SecDefaultAction) Running action: auditlog

[4] (SecDefaultAction) ignoring action: pass (rule contains a disruptive action)

[4] Running (non-disruptive) action: auditlog

[4] Running (disruptive)     action: deny

The debug log lists the rule id number for easy searching. In this example, the output is from our test rule with id number 1234.

Debug log configuration

By default, the debug log is disabled, as it can negatively affect performance. Just as with audit logging, the debug log is configured in /etc/nginx/modsec/modsecurity.conf. In that file, there are two configuration directives that are commented out. Uncomment them and change them to the below to enable debug logging:

SecDebugLog /var/log/modsec_debug.log
SecDebugLogLevel 9

• SecDebugLog — Specifies the path to the debug log file.
• SecDebugLogLevel — 0-9 value of how much information to log, with 9 being the highest. If you’re troubleshooting, setting this value to 9 is the most helpful.

Conclusion

In this blog post, we covered how to get started using the extensive logging capabilities within ModSecurity. ModSecurity has both audit logs, which contain information about all blocked transactions, and a debug log to further assist you if you’re having trouble using ModSecurity.

ModSecurity 3.0 is available for both the open source NGINX software and as the NGINX WAF for NGINX Plus. The NGINX WAF is a precompiled dynamic module that is maintained and fully supported by NGINX, Inc. Try it free for 30 days.

The post ModSecurity: Logging and debugging appeared first on NGINX.

Source: ModSecurity: Logging and debugging