Reports#

Reports are configurable summaries after a Robocop scan. For example, it can display a total number of issues discovered. They are dynamically loaded during setup according to a configuration.

Report class may collect rules messages from linter and parse it. At the end of the scan it will generate the report.

To enable report use -r / --reports argument and provide the name of the report. You can use multiple reports with separate arguments (-r report1 -r report2) or comma-separated list (-r report1,report2). Example:

robocop --reports rules_by_id,some_other_report path/to/file.robot

To enable all default reports use --reports all. Non-default reports can be only enabled using their name.

The order of the reports is preserved. For example, if you want timestamp report to be printed before any other reports, you can use the following configuration:

robocop --reports timestamp,all src.robot

List available reports#

Print a list of all reports with their configured status by using --list-reports:

robocop --reports all --list-reports

You can filter the list using optional ENABLED/DISABLED argument:

robocop --reports timestamp,sarif --list-reports DISABLED

Configuring reports#

Similarly as rules, reports can also be configured. The same --configure (or -c) option is used followed by report name, its parameter and the value:

robocop --configure <report_name>:<param_name>:<value>

For example:

robocop --configure return_status:quality_gate:E=100:W=100:I=9

configures quality_gate parameter and sets new threshold values for different severities of the rules (see more at Return status).

There are also other configurable reports like timestamp or json_report. More about them below.

Reports list#

Compare results#

Report name: compare_runs

Reports results can be compared with the previous run. Example output:

Found 18 (-3) issues: 13 (-4) INFOs, 5 (+1) WARNINGs.

Issues by ID:
I0923 (unnecessary-string-conversion)     : 10 (+0)
W0922 (variable-overwritten-before-usage) : 2 (+1)
I0920 (unused-variable)                   : 2 (-4)
W0301 (not-allowed-char-in-name)          : 2 (+0)
W0324 (overwriting-reserved-variable)     : 1 (+0)
I0605 (could-be-test-tags)                : 1 (+0)

Robocop stores previous result in cache directory. Cache directory is stored in the different location depending on the platform:

Linux: "~/.cache/robocop"
macOS: "~/Library/Caches/robocop"
Windows: "C:\\Users\\<username>\\AppData\\Local\\robocop"

Saving the results is disabled by default and can be enabled with --persistent flag:

robocop --persistent

or in the TOML configuration file:

[tool.robocop]
persistent = true

Only the previous run for the current working directory is saved.

To used stored results to compare with current run, enable compare_runs report:

robocop --reports all,compare_runs

Rules by ID#

Report name: rules_by_id

Report that groups linter rules messages by rule id and prints it ordered by most common message. Example:

Issues by ID:
W0502 (too-little-calls-in-keyword) : 5
W0201 (missing-doc-keyword)         : 4
E0401 (parsing-error)               : 3
W0301 (not-allowed-char-in-name)    : 2
W0901 (keyword-after-return)        : 1

Rules by severity#

Report name: rules_by_error_type

Report that groups linter rules messages by severity and prints total of issues per every severity level.

Example:

Found 15 issues: 4 ERRORs, 11 WARNINGs.

Return status#

Report name: return_status

This report is always enabled. Report that checks if number of returned rules messages for given severity value does not exceed preset threshold. That information is later used as a return status from Robocop.

Execution time#

Report name: scan_timer

Report that returns Robocop execution time

Example:

Scan finished in 0.054s.

File statistics#

Report name: file_stats

Report that displays overall statistics about number of processed files.

Example:

Processed 7 files from which 5 files contained issues.

Robocop version#

Report name: version

Report that returns Robocop version.

Example:

Report generated by Robocop version: 2.0.2

Report timestamp#

Report name: timestamp

Report that returns Robocop execution timestamp. Timestamp follows local time in format of Year-Month-Day Hours(24-hour clock):Minutes:Seconds ±hh:mm UTC offset as default.

Example:

Reported: 2022-07-10 21:25:00 +0300

Both of default values, timezone and format can be configured by -c/--configure and timestamp:timezone:"<timezone name>" and/or timestamp:format:"<format string>":

robocop --configure timestamp:timezone:"Europe/Paris" --configure timestamp:format:"%Y-%m-%d %H:%M:%S %Z %z"

This yields following timestamp report:

Reported: 2022-07-10 20:38:10 CEST +0200

For timezone names, see here.

For timestamp formats, see datetime format codes.

Useful configurations:

Local time to ISO 8601 format:
robocop --configure timestamp:format:"%Y-%m-%dT%H:%M:%S%z"

UTC time:
robocop --configure timestamp:timezone:"UTC" --configure timestamp:format:"%Y-%m-%dT%H:%M:%S %Z %z"

Timestamp with high precision:
robocop --configure timestamp:format:"%Y-%m-%dT%H:%M:%S.%f %z"

12-hour clock:
robocop --configure timestamp:format:"%Y-%m-%d %I:%M:%S %p %Z %z"

More human-readable format 'On 10 July 2022 07:26:24 +0300':
robocop --configure timestamp:format:"On %d %B %Y %H:%M:%S %z"

JSON export#

Report name: json_report

Report that returns a list of found issues in a JSON format. The output file will be generated in the current working directory with the robocop.json name.

This report is not included in the default reports. The --reports all option will not enable this report. You can still enable it using report name directly: --reports json_report or --reports all,json_report.

You can configure output directory and report filename:

robocop --configure json_report:output_dir:C:/json_reports
robocop --configure json_report:report_filename:robocop_output.json

Example content of the file:

[
    {
        "source": "C:\robot_tests\keywords.robot",
        "line": 1,
        "end_line": 1,
        "column": 1,
        "end_column": 1,
        "severity": "I",
        "rule_id": "0913",
        "description": "No tests in 'keywords.robot' file, consider renaming to 'keywords.resource'",
        "rule_name": "can-be-resource-file"
    },
    {
        "source": "C:\robot_tests\keywords.robot",
        "line": 51,
        "end_line": 51,
        "column": 1,
        "end_column": 13,
        "severity": "W",
        "rule_id": "0324",
        "description": "Variable '${TEST_NAME}' overwrites reserved variable '${TEST_NAME}'",
        "rule_name": "overwriting-reserved-variable"
    }
]

SARIF export#

Report name: sarif

Report that generates SARIF output file.

This report is not included in the default reports. The --reports all option will not enable this report. You can still enable it using report name directly: --reports sarif or --reports all,sarif.

All fields required by GitHub Code Scanning are supported. The output file will be generated in the current working directory with the .sarif.json name.

You can configure output directory and report filename:

robocop --configure sarif:output_dir:C:/sarif_reports --configure sarif:report_filename:.sarif