Reports#
Reports are configurable summaries after a Robocop scan. For example, it can display a total number of issues discovered. They are dynamically loaded during setup according to a configuration.
Report class may collect rules messages from linter and parse it. At the end of the scan it will generate the report.
To enable report use -r
/ --reports
argument and provide the name of the report.
You can use multiple reports with separate arguments (-r report1 -r report2
) or comma-separated list
(-r report1,report2
). Example:
robocop --reports rules_by_id,some_other_report path/to/file.robot
To enable all default reports use --reports all
. Non-default reports can be only enabled using their name.
The order of the reports is preserved. For example, if you want timestamp
report to be printed before any
other reports, you can use the following configuration:
robocop --reports timestamp,all src.robot
Print a list of all reports with their configured status by using --list-reports
:
robocop --reports all --list-reports
You can filter the list using optional ENABLED
/DISABLED
argument:
robocop --reports timestamp,sarif --list-reports DISABLED
Compare report results#
Reports results can be compared with the previous run. Example output:
Found 18 (-3) issues: 13 (-4) INFOs, 5 (+1) WARNINGs.
Issues by ID:
I0923 (unnecessary-string-conversion) : 10 (+0)
W0922 (variable-overwritten-before-usage) : 2 (+1)
I0920 (unused-variable) : 2 (-4)
W0301 (not-allowed-char-in-name) : 2 (+0)
W0324 (overwriting-reserved-variable) : 1 (+0)
I0605 (could-be-test-tags) : 1 (+0)
Robocop stores previous result in cache directory. Cache directory is stored in the different location depending on the platform:
- Linux: "~/.cache/robocop"
- macOS: "~/Library/Caches/robocop"
- Windows: "C:\\Users\\<username>\\AppData\\Local\\robocop"
Saving the results is disabled by default and can be enabled with --persistent
flag:
robocop --persistent
or in the TOML configuration file:
[tool.robocop]
persistent = true
Only the previous run for the current working directory is saved.
To used stored results to compare with current run, enable compare_runs
report:
robocop --reports all,compare_runs
- class robocop.reports.rules_by_id_report.RulesByIdReport(compare_runs)#
Report name:
rules_by_id
Report that groups linter rules messages by rule id and prints it ordered by most common message. Example:
Issues by ID: W0502 (too-little-calls-in-keyword) : 5 W0201 (missing-doc-keyword) : 4 E0401 (parsing-error) : 3 W0301 (not-allowed-char-in-name) : 2 W0901 (keyword-after-return) : 1
- class robocop.reports.rules_by_severity_report.RulesBySeverityReport(compare_runs)#
Report name:
rules_by_error_type
Report that groups linter rules messages by severity and prints total of issues per every severity level.
Example:
Found 15 issues: 4 ERRORs, 11 WARNINGs.
- class robocop.reports.return_status_report.ReturnStatusReport#
Report name:
return_status
This report is always enabled. Report that checks if number of returned rules messages for given severity value does not exceed preset threshold. That information is later used as a return status from Robocop.
- class robocop.reports.time_taken_report.TimeTakenReport(compare_runs)#
Report name:
scan_timer
Report that returns Robocop execution time
Example:
Scan finished in 0.054s.
- class robocop.reports.file_stats_report.FileStatsReport(compare_runs)#
Report name:
file_stats
Report that displays overall statistics about number of processed files.
Example:
Processed 7 files from which 5 files contained issues.
- class robocop.reports.robocop_version_report.RobocopVersionReport(compare_runs)#
Report name:
version
Report that returns Robocop version.
Example:
Report generated by Robocop version: 2.0.2
- class robocop.reports.timestamp_report.TimestampReport#
Report name:
timestamp
Report that returns Robocop execution timestamp. Timestamp follows local time in format of Year-Month-Day Hours(24-hour clock):Minutes:Seconds ±hh:mm UTC offset as default.
Example:
Reported: 2022-07-10 21:25:00 +0300
Both of default values,
timezone
andformat
can be configured by-c/--configure
andtimestamp:timezone:"<timezone name>"
and/ortimestamp:format:"<format string>"
:robocop --configure timestamp:timezone:"Europe/Paris" --configure timestamp:format:"%Y-%m-%d %H:%M:%S %Z %z"
This yields following timestamp report:
Reported: 2022-07-10 20:38:10 CEST +0200
For timezone names, see: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
For timestamp formats, see: https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
Useful configurations:
Local time to ISO 8601 format: robocop --configure timestamp:format:"%Y-%m-%dT%H:%M:%S%z" UTC time: robocop --configure timestamp:timezone:"UTC" --configure timestamp:format:"%Y-%m-%dT%H:%M:%S %Z %z" Timestamp with high precision: robocop --configure timestamp:format:"%Y-%m-%dT%H:%M:%S.%f %z" 12-hour clock: robocop --configure timestamp:format:"%Y-%m-%d %I:%M:%S %p %Z %z" More human-readable format 'On 10 July 2022 07:26:24 +0300': robocop --configure timestamp:format:"On %d %B %Y %H:%M:%S %z"
- class robocop.reports.json_report.JsonReport#
Report name:
json_report
Report that returns a list of found issues in a JSON format. The output file will be generated in the current working directory with the
robocop.json
name.This report is not included in the default reports. The
--reports all
option will not enable this report. You can still enable it using report name directly:--reports json_report
or--reports all,json_report
.You can configure output directory and report filename:
robocop --configure json_report:output_dir:C:/json_reports robocop --configure json_report:report_filename:robocop_output.json
Example content of the file:
[ { "source": "C:\robot_tests\keywords.robot", "line": 1, "end_line": 1, "column": 1, "end_column": 1, "severity": "I", "rule_id": "0913", "description": "No tests in 'keywords.robot' file, consider renaming to 'keywords.resource'", "rule_name": "can-be-resource-file" }, { "source": "C:\robot_tests\keywords.robot", "line": 51, "end_line": 51, "column": 1, "end_column": 13, "severity": "W", "rule_id": "0324", "description": "Variable '${TEST_NAME}' overwrites reserved variable '${TEST_NAME}'", "rule_name": "overwriting-reserved-variable" } ]
- class robocop.reports.sarif_report.SarifReport#
Report name:
sarif
Report that generates SARIF output file.
This report is not included in the default reports. The
--reports all
option will not enable this report. You can still enable it using report name directly:--reports sarif
or--reports all,sarif
.All fields required by GitHub Code Scanning are supported. The output file will be generated in the current working directory with the
.sarif.json
name.You can configure output directory and report filename:
robocop --configure sarif:output_dir:C:/sarif_reports --configure sarif:report_filename:.sarif