Reports

Reports are configurable summaries after a Robocop scan. For example, it can display a total number of issues discovered. They are dynamically loaded during setup according to a configuration.

Report class may collect rules messages from linter and parse it. At the end of the scan it will generate the report.

To enable report use -r / --reports argument and provide the name of the report. You can use multiple reports with separate arguments (-r report1 -r report2) or comma-separated list (-r report1,report2). Example:

robocop --reports rules_by_id,some_other_report path/to/file.robot

To enable all default reports use --reports all. Non-default reports can be only enabled using their name.

The order of the reports is preserved. For example, if you want timestamp report to be printed before any other reports, you can use the following configuration:

robocop --reports timestamp,all src.robot

Print a list of all reports with their configured status by using --list-reports:

robocop --reports all --list-reports

You can filter the list using optional ENABLED/DISABLED argument:

robocop --reports timestamp,sarif --list-reports DISABLED

Compare report results

Reports results can be compared with the previous run. Example output:

Found 18 (-3) issues: 13 (-4) INFOs, 5 (+1) WARNINGs.

Issues by ID:
I0923 (unnecessary-string-conversion)     : 10 (+0)
W0922 (variable-overwritten-before-usage) : 2 (+1)
I0920 (unused-variable)                   : 2 (-4)
W0301 (not-allowed-char-in-name)          : 2 (+0)
W0324 (overwriting-reserved-variable)     : 1 (+0)
I0605 (could-be-test-tags)                : 1 (+0)

Robocop stores previous result in cache directory. Cache directory is stored in the different location depending on the platform:

- Linux: "~/.cache/robocop"
- macOS: "~/Library/Caches/robocop"
- Windows: "C:\\Users\\<username>\\AppData\\Local\\robocop"

Saving the results is disabled by default and can be enabled with --persistent flag:

robocop --persistent

or in the TOML configuration file:

[tool.robocop]
persistent = true

Only the previous run for the current working directory is saved.

To used stored results to compare with current run, enable compare_runs report:

robocop --reports all,compare_runs

class robocop.reports.rules_by_id_report.RulesByIdReport(compare_runs)

Report name: rules_by_id

Report that groups linter rules messages by rule id and prints it ordered by most common message. Example:

Issues by ID:
W0502 (too-little-calls-in-keyword) : 5
W0201 (missing-doc-keyword)         : 4
E0401 (parsing-error)               : 3
W0301 (not-allowed-char-in-name)    : 2
W0901 (keyword-after-return)        : 1

class robocop.reports.rules_by_severity_report.RulesBySeverityReport(compare_runs)

Report name: rules_by_error_type

Report that groups linter rules messages by severity and prints total of issues per every severity level.

Example:

Found 15 issues: 4 ERRORs, 11 WARNINGs.

class robocop.reports.return_status_report.ReturnStatusReport

Report name: return_status

This report is always enabled. Report that checks if number of returned rules messages for given severity value does not exceed preset threshold. That information is later used as a return status from Robocop.

class robocop.reports.time_taken_report.TimeTakenReport(compare_runs)

Report name: scan_timer

Report that returns Robocop execution time

Example:

Scan finished in 0.054s.

class robocop.reports.file_stats_report.FileStatsReport(compare_runs)

Report name: file_stats

Report that displays overall statistics about number of processed files.

Example:

Processed 7 files from which 5 files contained issues.

class robocop.reports.robocop_version_report.RobocopVersionReport(compare_runs)

Report name: version

Report that returns Robocop version.

Example:

Report generated by Robocop version: 2.0.2

class robocop.reports.timestamp_report.TimestampReport

Report name: timestamp

Report that returns Robocop execution timestamp. Timestamp follows local time in format of Year-Month-Day Hours(24-hour clock):Minutes:Seconds ±hh:mm UTC offset as default.

Example:

Reported: 2022-07-10 21:25:00 +0300

Both of default values, timezone and format can be configured by -c/--configure and timestamp:timezone:"<timezone name>" and/or timestamp:format:"<format string>":

robocop --configure timestamp:timezone:"Europe/Paris" --configure timestamp:format:"%Y-%m-%d %H:%M:%S %Z %z"

This yields following timestamp report:

Reported: 2022-07-10 20:38:10 CEST +0200

For timezone names, see: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

For timestamp formats, see: https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes

Useful configurations:

Local time to ISO 8601 format:
robocop --configure timestamp:format:"%Y-%m-%dT%H:%M:%S%z"

UTC time:
robocop --configure timestamp:timezone:"UTC" --configure timestamp:format:"%Y-%m-%dT%H:%M:%S %Z %z"

Timestamp with high precision:
robocop --configure timestamp:format:"%Y-%m-%dT%H:%M:%S.%f %z"

12-hour clock:
robocop --configure timestamp:format:"%Y-%m-%d %I:%M:%S %p %Z %z"

More human-readable format 'On 10 July 2022 07:26:24 +0300':
robocop --configure timestamp:format:"On %d %B %Y %H:%M:%S %z"

class robocop.reports.json_report.JsonReport

Report name: json_report

Report that returns a list of found issues in a JSON format. The output file will be generated in the current working directory with the robocop.json name.

This report is not included in the default reports. The --reports all option will not enable this report. You can still enable it using report name directly: --reports json_report or --reports all,json_report.

You can configure output directory and report filename:

robocop --configure json_report:output_dir:C:/json_reports
robocop --configure json_report:report_filename:robocop_output.json

Example content of the file:

[
    {
        "source": "C:\robot_tests\keywords.robot",
        "line": 1,
        "end_line": 1,
        "column": 1,
        "end_column": 1,
        "severity": "I",
        "rule_id": "0913",
        "description": "No tests in 'keywords.robot' file, consider renaming to 'keywords.resource'",
        "rule_name": "can-be-resource-file"
    },
    {
        "source": "C:\robot_tests\keywords.robot",
        "line": 51,
        "end_line": 51,
        "column": 1,
        "end_column": 13,
        "severity": "W",
        "rule_id": "0324",
        "description": "Variable '${TEST_NAME}' overwrites reserved variable '${TEST_NAME}'",
        "rule_name": "overwriting-reserved-variable"
    }
]

class robocop.reports.sarif_report.SarifReport

Report name: sarif

Report that generates SARIF output file.

This report is not included in the default reports. The --reports all option will not enable this report. You can still enable it using report name directly: --reports sarif or --reports all,sarif.

All fields required by GitHub Code Scanning are supported. The output file will be generated in the current working directory with the .sarif.json name.

You can configure output directory and report filename:

robocop --configure sarif:output_dir:C:/sarif_reports --configure sarif:report_filename:.sarif