How do I include failed crawler requests in Matomo log analytics?

Definition: Matomo log analytics imports website access logs into analytics reports. As of October 2026, combining --enable-bots and --enable-http-errors includes crawler requests that return HTTP errors, with the response status stored in the HTTP-code custom variable.

Key facts

• Matomo's additional-data import documentation says --enable-http-errors records responses with 4xx or 5xx status as page views, including codes such as 404 and 500.
• The importer source code assigns HTTP-code to individual page views and, when bot import is enabled, assigns the Bot custom variable to crawler visits.
• Matomo's Custom Variables guide places the report under Visitors > Custom Variables. Its table expands variable names to show their recorded values.

Which import command includes failed crawler requests?

Matomo's default-options guide locates the On-Premise importer at misc/log-analytics/import_logs.py. The following command adapts its example to include bots and HTTP errors:

python /path/to/matomo/misc/log-analytics/import_logs.py \
  --url=https://analytics.example.com \
  --idsite=1234 \
  --enable-bots \
  --enable-http-errors \
  /path/to/your/web/access.log

The script path, analytics URL, site ID and access-log path are placeholders. The same guide requires --token-auth=SECRET_TOKEN_HERE when the importer runs on a different server from Matomo. It also documents --debug for troubleshooting log-format selection.

Bot identification uses the log's User-agent field, according to the additional-data documentation. The importer normally excludes bot traffic, so enabling HTTP errors alone leaves that exclusion in place. Enabling bots also sends their page views to Matomo and increases processing time.

Where are response codes reported?

A custom variable is a named value attached to a visit or page view. Matomo's Custom Variables guide explains that its report lists names with expandable values. Expanding HTTP-code therefore exposes the imported response-code values. For page-scoped variables, the report provides the Actions metric; visit-level metrics such as Visits and Unique Visitors appear for visit-scoped variables.

The same guide says Custom Variables are available by default in Matomo Cloud. On-Premise installations require installation and activation of the Custom Variables plugin through the Marketplace, using a Super User account.

The importer implementation stores the crawler's user-agent string as the value of the visit-level Bot variable. For crawler-focused analysis, that recorded variable supplies a segment condition. Matomo's segment editor guide documents selecting a dimension, comparison operator and value, then using TEST to inspect matching visits before Save & Apply.

Why can some requests still be missing?

Static-file requests have a separate import setting. Matomo's additional-data documentation assigns --enable-static to images, JavaScript and CSS. Redirects have another switch, --enable-http-redirects; the same documentation says 304 responses are already tracked as page views.

The importer source also defines --dry-run for a trial without inserting tracking data. Matomo's default-options guide recommends importing into a test site first because the importer does not check for duplicate data. Reimporting the same logs can duplicate the destination data.

Related terms

• How do I include crawler requests in a Matomo log import?
• How do I import website access logs into Matomo?
• HTTP status codes
• Server log analysis

FAQ

Does enabling bots also enable HTTP errors?

You need both switches for failed crawler requests. Matomo's additional-data documentation documents bot inclusion and HTTP-error inclusion as separate options.

Can I see which page linked to an error?

If your access log supplies a referrer URL, Matomo's additional-data documentation says the imported error page view includes that referrer in its page title.

Why is the Custom Variables report missing?

If you use Matomo On-Premise, the plugin guide says Custom Variables are no longer installed by default. Install and activate the plugin to use its report.

Sources

1. Matomo, "Import additional data including bots, static files, and HTTP errors tracking", undated
2. Matomo, "How do I run the Log File importer script with default options?", undated
3. Matomo, "Custom Variables", undated
4. Matomo, "import_logs.py", undated
5. Matomo, "Build custom segments in Matomo", undated

---

Check how a page reads to answer engines. Open the GEO score analyzer