Tools and Utilities

To run the Python utilities (those ending in .py), you will need to have the PYTHONPATH environment variable set. This can be accomplished one of two ways: by prefixing all commands with a local variable (PYTHONPATH=/path/to/scap-security-guide), or by exporting PYTHONPATH in your shell environment. We provide a script for making this easier: .pyenv.sh. To set PYTHONPATH correctly for the current shell, simply call source .pyenv.sh. For more information on how to use this script, please see the comments at the top of the file.

Profile Statistics and Utilities

The profile_tool.py tool displays XCCDF profile statistics. It can show number of rules in the profile, how many of these rules have an OVAL check implemented, how many have a remediation available, shows rule IDs which are missing them and other useful information.

To use the script, first build the content, then pass the built XCCDF (not data stream) to the script.

For example, to check which rules in RHEL8 OSPP profile are missing remediations, run this command:

    $ ./build_product rhel8
    $ ./build-scripts/profile_tool.py stats --missing-fixes --profile ospp --benchmark build/ssg-rhel8-xccdf.xml

Note: There is an automated job which provides latest statistics from all products and all profiles, you can view it here: Statistics

The tool also can subtract rules between YAML profiles.

For example, to subtract selected rules from a given profile based on rules selected by another profile, run this command:

    $ ./build-scripts/profile_tool.py sub --profile1 rhel9/profiles/ospp.profile --profile2 rhel9/profiles/pci-dss.profile

This will result in a new YAML profile containing exclusive rules to the profile pointed by the --profile1 option.

The tool can also generate a list of the most used rules contained in profiles from a given data stream or benchmark.

For example, to get a list of the most used rules in the benchmark for rhel8, run this command:

    $ ./build-scripts/profile_tool.py most-used-rules build/ssg-rhel8-xccdf.xml

Or you can also run this command to get a list of the most used rules in the entire project:

    $ ./build-scripts/profile_tool.py most-used-rules

Optionally, you can use this command to limit the statistics for a specific product:

    $ ./build-scripts/profile_tool.py most-used-rules --products rhel9

The result will be a list of rules with the number of uses in the profiles. The list can be generated as plain text, JSON or CVS. Via the --format FORMAT parameter.

The tool can also generate a list of the most used component based on rules contained in profiles from the entire project:

    $ ./build-scripts/profile_tool.py most-used-components

Optionally, you can use this command to limit the statistics for a specific product:

    $ ./build-scripts/profile_tool.py most-used-components --products rhel9

You can also get a list of the most used components with used rules for the RHEL9 product, you can use the --rules flag. As shown in this command:

    $ ./build-scripts/profile_tool.py most-used-components --products rhel9 --rules

You can also use the --all flag to get a list of all components and rules in the output, including unused components and unused rules. As shown in this command:

    $ ./build-scripts/profile_tool.py most-used-components --products rhel9 --all

The result will be a list of rules with the number of uses in the profiles. The list can be generated as plain text, JSON or CVS. Via the --format FORMAT parameter.

Generating Controls from DISA’s XCCDF Files

If you want a control file for product from DISA’s XCCDF files you can run the following command: It supports the following arguments:

options:
  -h, --help            show this help message and exit
  -r ROOT, --root ROOT  Path to SSG root directory (defaults to the root of the repository)
  -o OUTPUT, --output OUTPUT
                        File to write yaml output to (defaults to build/stig_control.yml)
  -p PRODUCT, --product PRODUCT
                        What product to get STIGs for
  -m MANUAL, --manual MANUAL
                        Path to XML XCCDF manual file to use as the source of the STIGs
  -j JSON, --json JSON  Path to the rules_dir.json (defaults to build/rule_dirs.json)
  -c BUILD_CONFIG_YAML, --build-config-yaml BUILD_CONFIG_YAML
                        YAML file with information about the build configuration.
  -ref REFERENCE, --reference REFERENCE
                        Reference system to check for, defaults to stigid
  -s, --split           Splits the each ID into its own file.

Example

    $ ./utils/build_stig_control.py -p rhel8 -m shared/references/disa-stig-rhel8-v1r5-xccdf-manual.xml

Generating Controls From a Reference

When converting profile to use a control file this script can be helpful in creating the skeleton control. The output of this script will need to be adjusted to add other keys such as title or description to the controls. This script does require that ./utils/rule_dir_json.py be run before this script is used. See ./utils/build_control_from_reference.py --help for the full set options the script provides.

Example

    $ ./utils/build_control_from_reference.py --product rhel10 --reference ospp --output controls/ospp.yml

Generating login banner regular expressions

Rules like banner_etc_issue and dconf_gnome_login_banner_text will check for configuration of login banners and remediate them. Both rules source the banner text from the same variable login_banner_text, and the banner texts need to be in the form of a regular expression. There are a few utilities you can use to transform your text into the appropriate regular expression:

When adding a new banner directly to the login_banner_text, use the custom Jinja filter banner_regexify. If customizing content via SCAP Workbench, or directly writing your tailoring XML, use utils/regexify_banner.py to generate the appropriate regular expression.

Modifying rule directory content files

All utilities discussed below require information about the existing rules for fast operation. We’ve provided the utils/rule_dir_json.py script to build this information in a format understood by these scripts.

To execute it:

    $ ./utils/rule_dir_json.py

Optionally, provide a path to a CaC root and destination YAML file:

    $ ./utils/rule_dir_json.py --root /path/to/ComplianceAsCode/content \
                               --output /tmp/rule_dirs.json

`utils/fix_rules.py` – automatically fix-up rules

utils/fix_rules.py includes various sub-commands for automatically fixing common problems in rules.

These sub-commands are:

empty_identifiers: removes any identifiers which are empty.
invalid_identifiers: removes any invalid CCE identifiers (due to incorrect format).
int_identifiers: turns any identifiers which are an integer into a string.
empty_references: removes any references which are empty.
int_references: turns any references which are an integer into a string.
duplicate_subkeys: finds (but doesn’t fix!) any rules with duplicated identifiers or references.
sort_subkeys: sorts all subkeys under identifiers and references.
add-cce: automatically assign CCE identifiers to rules.

To execute:

    $ ./utils/fix_rules.py [--assume-yes] [--dry-run] <command>

For example:

    $ ./utils/fix_rules.py -y sort_subkeys

Note that it is generally good practice to commit all changes prior to running one of these commands and then commit the results separately.

How to automatically assign CCEs with the `add-cce` sub-command

First, you need to make sure that that the rule_dirs.json exists, run the following command to create it:

    $ ./utils/rule_dir_json.py

Then based on the available pool you want to assign the CCEs, you can run something like:

    $ python utils/fix_rules.py --product products/rhel9/product.yml add-cce --cce-pool redhat audit_rules_privileged_commands_newuidmap

Note: Multiple rules can have the CCE at the same time by just adding space separated rule IDs.

Example for sle15 product:

    $ python utils/fix_rules.py --product products/sle15/product.yml add-cce --cce-pool sle15 audit_rules_privileged_commands_newuidmap audit_rules_privileged_commands_newuidmap

`utils/refchecker.py` – automatically check `rule.yml` for references

This utility checks all rule.yml referenced from a given profile for the specified reference. Unlike build-scripts/profile_tool.py, which operates on the built XCCDF information, utils/refchecker.py operates on the contents of the rule.yml files.

To execute:

    $ ./utils/refchecker.py <product> <profile> <reference>

For example:

    $ ./utils/refchecker.py ubuntu2004 cis_level1_server cis

This utility has some knowledge of which references are product-specific (checking for cis@ubuntu2004 in the above example) and which are product-independent.

Note that this utility does not modify the rule directories at all.

`utils/mod_checks.py` and `utils/mod_fixes.py` – programmatically modify check and fix applicability

These two utilities have identical usage. Both modifies the platform/product applicability of various files (either OVAL or hardening content). They support the following sub-commands:

add: add the given platform(s) to the specified rule’s OVAL check. Note: Only applies to shared content.
list: list the given OVAL(s) and the products that apply to them; empty if product-independent.
remove: remove the given platform(s) from the specified rule’s OVAL check. Note: Only applies to shared content.
replace: perform a pattern-match replacement on the specified rule’s platform applicability. Note: Only applies to shared content.
diff: perform a textual diff between content for the specified products.
delete: remove an OVAL for the specified product.
make_shared: move a product-specific OVAL into a shared OVAL.

To execute:

    $ ./utils/mod_checks.py <rule_id> <command> [...other arguments...]
    $ ./utils/mod_fixes.py <rule_id> <type> <command> [...other arguments...]

For an example of add:

    $ ./utils/mod_checks.py clean_components_post_updating add multi_platform_sle
    $ ./utils/mod_fixes.py clean_components_post_updating bash add multi_platform_sle

For an example of list:

    $ ./utils/mod_checks.py clean_components_post_updating list
    $ ./utils/mod_fixes.py clean_components_post_updating ansible list

For an example of remove:

    $ ./utils/mod_checks.py file_permissions_local_var_log_messages remove multi_platform_sle
    $ ./utils/mod_fixes.py file_permissions_local_var_log_messages bash remove multi_platform_sle

For an example of replace:

    $ ./utils/mod_checks.py file_permissions_local_var_log_messages replace multi_platform_sle~multi_platform_sle,multi_platform_ubuntu
    $ ./utils/mod_fixes.py file_permissions_local_var_log_messages bash replace multi_platform_sle~multi_platform_sle,multi_platform_ubuntu

For an example of diff:

    $ ./utils/mod_checks.py clean_components_post_updating diff sle12 sle15
    $ ./utils/mod_fixes.py clean_components_post_updating bash diff sle12 sle15

For an example of delete:

    $ ./utils/mod_checks.py clean_components_post_updating delete sle12
    $ ./utils/mod_fixes.py clean_components_post_updating bash delete sle12

For an example of make_shared:

    $ ./utils/mod_checks.py clean_components_post_updating make_shared sle12
    $ ./utils/mod_fixes.py clean_components_post_updating bash make_shared sle12

`utils/create_scap_delta_tailoring.py` – Create tailoring files for rules not covered by other content

The goal of this tool is to create a tailoring file that enable rules that are not covered by other SCAP content and disables rules that are covered by the given content. It supports the following arguments:

-r, --root - Path to SSG root directory
-p, --product - What product to produce the tailoring file for (required)
-m, --manual - Path to the XCCDF XML file of the SCAP content (required)
-j, --json - Path to the rules_dir.json file.
- Defaults to build/stig_control.json
-c, --build-config-yaml - YAML file with information about the build configuration.
- Defaults to build/build_config.yml
-b, --profile - What profile to use.
- Defaults to stig
-ref, --reference - What reference system to check for.
- Defaults to stigid
- -o, --output - Defaults build/PRODUCT_PROFILE_tailoring.xml, where PRODUCT and PROFILE are respective parameters given to the script.
- --profile-id - The id of the created profile. Defaults to PROFILE_delta
- --tailoring-id - The id of the created tailoring file. Defaults to xccdf_content-disa-delta_tailoring_default

To execute:

    $ ./utils/create_scap_delta_tailoring.py -p rhel8 -b stig -m shared/references/disa-stig-rhel8-v1r4-xccdf-scap.xml

`utils/compare_ds.py` – Compare two data streams (can also compare XCCDFs)

This script compares two data streams or two benchmarks and generates a diff output. It can show what changed in rules, for example in description, references and remediation scripts. Changes in checks (OVAL and OCIL) are shown too, but the OVAL diff is limited to the criteria and criterion order and their IDs.

When comparing contents from DISA, either the Automated content or Manual benchmark, use the --disa-content option, this options maps the rules in the old content to the rule in the new content. The rule mapping is necessary because the IDs in DISA’s content’s rules change IDs whenever they are updated.

By default, the script prints the diff to the standard output, which can generate a single huge diff for the whole data stream or benchmark. The option --rule-diffs can be used to generate a diff file per rule. In this mode the diff files are created in a directory: ./compare_ds-diffs. To change the output dir use --output-dir option.

Compare current DISA’s manual benchmark, and generate per file diffs:

    $ utils/compare_ds.py --disa-content --rule-diffs ./disa-stig-rhel8-v1r6-xccdf-manual.xml shared/references/disa-stig-rhel8-v1r7-xccdf-manual.xml

Compare two data streams:

    $ utils/compare_ds.py /tmp/ssg-rhel8-ds.xml build/ssg-rhel8-ds.xml > content.diff

HTML Diffs

The diffs generated by utils/compare_ds.py can be transformed to HTML diffs with diff2html utility.

Install diff2html:

    # Fedora
    $ sudo dnf install npm
    $ sudo npm install -g diff2html-cli

Generate the HTML diffs:

    $ mkdir -p html
    $ for f in $(ls compare_ds-diffs/); do diff2html -i file -t $f -F "html/$f.html" "compare_ds-diffs/$f"; done

`utils/compare_results.py` – Compare to two ARF result files

The goal of this script is to compare the result of two ARF files. It will show what rules are missing, different, and the same between the two files. The script can take results from content created by this repo and by DISA. If the result files come from the same source the script will use XCCDF ids as basis for the comparison. Otherwise, the script will use STIG ids to compare.

If one STIG ID has more than one result (this is the case for a few STIG IDs in this repo) the results will be merged. Given a set of status the script will select the status from the group that is the highest value on the list below.

Error
Fail
Not applicable
Not selected
Not checked
Informational
Pass

Examples:

[pass, pass] will result in pass
[pass, fail] will result in fail
[pass, error, fail] will result in error

To execute:

    $ ./utils/compare_results.py ssg_results.xml disa_results.xml

`utils/import_srg_spreadsheet.py` – Import changes made to an SRG Spreadsheet into the project

This script will import changes from a SRG export spreadsheet. This script is designed to be run then each file reviewed carefully before being committed.

It supports the following arguments:

-b, --current — Path to the current XLSX export (required)
-c, --changed — Path to the XLSX that was changed, defaults to RHEL 9
-e, --end-row — What row to stop scanning, defaults to 600.
-j, --json — Path to the rules_dir.json file.
-n, --changed-name — The name of the current in the changed file (required)
-p, --product — What product to produce the tailoring file for (required)
-r, --root — Path to SSG root directory

To execute:

    $ ./utils/import_srg_spreadsheet.py --changed 20220811_submission.xlsx --current build/cac_stig_output.xlsx -p rhel9

`utils/import_disa_stig.py` – Import Content from DISA’s XML Content

This script imports SRG Requirements, Check Text, and Fix Text from a DISA STIG XML File. This script only updates STIG items that only have one rule assigned to them.

To execute:

$ ./utils/import_disa_stig.py --product rhel9 --control stig_rhel9 shared/references/disa-stig-rhel9-v1r2-xccdf-manual.xml

Profiling the Build System

The goal of utils/build_profiler.sh and utils/build_profiler_report.py is to help developers measure and compare build times of a single product or a group of products and determine what impact their changes had on the speed of the buildsystem. Both of these tools shouldn’t be invoked alone but rather through the build_product script by using the -p|–profiling switch.

The intended usage is:

    $ ./build_product <products> -p|--profiling

`utils/build_profiler.sh` – Handle directory structure for profiling files and invokes other script

The goal of this tool is to create the directory structure necessary for the profiling system and create a new numbered logfile, as well as invoking the utils/build_profiler_report.py and subsequently generating an interactive HTML file using webtreenode.

It is invoked by the build_product script. When invoked for the first time, it creates the .build_profiling directory and then a directory inside it named product_string, which is passed from the build_product script. This is done so that each product combination being built has its own directory for log files, because different combinations may affect each other and have different build times.

The script then moves the ninja log from the build folder to the product_string folder and numbers it. The baseline script is number 0 and if missing, the script will call the utils/build_profiler_report.py script with the --baseline switch.

It then invokes the utils/build_profiler_report.py script with a logfile name, as well as the optional baseline switch. After that, it checks if the .webtreenode file was generated and then uses the webtreenode command to generate an interactive HTML report.

It supports exactly one argument:

"product_string" - Contains all the product names that were built joined by underscores

To execute:

    $ ./build_profiler.sh <product_string>

`utils/build_profiler_report.py` – Parse a ninja file and display report to user

The goal of this tool is to generate a report about differences in build times, both a text version in the terminal and a webtreenode version that is later converted into an interactive HTML report.

The script parses the data from "logfile" as the current logfile and the data from 0.ninja_log as the baseline logfile (if the --baseline switch is used, the baseline log is not loaded). It then generates a .webtreemap file and prints the report:

Target - The target/outputfile that was built
% - The percentage of the total build time that the target took
D - The difference of the % percentage between baseline and current logfile (dimensionless value, not a percentage) - This is the most important metric, as it signifies the ratio of this target to the other targets, therefore it shouldn’t be affected too much by the speed of the hardware
T - The time that the target took to get built in an h:m:s format
TD - The time difference between baseline and current logfile in an h:m:s format
%TD - The percentage difference of build times between current and baseline logfile

It supports up to two arguments:

"logfile" - [mandatory] Name of the numbered ninja logfile, e.g. 0.ninja_log
--baseline - [optional] If the switch is used, the values are not compared with a baseline log

To execute:

    $ ./build_profiler_report.py <logfile> [--baseline]

Other Scripts

`utils/srg_diff.py` – Compare Two SRG Spreadsheets

This script will output an HTML page that compares two SRG exports. This script should help with reviewing changes created by utils/import_srg_spreadsheet.py script. This script assumes that the STIG ID columns are compatible. This script needs the project built for the given product and utils/rule_dir_json.py ran.

The report has the following sections:

Missing in base/target: These sections list rules are not in the other spreadsheet
Rows Missing STIG ID in base/target: These section list the requirement and SRG IDs of rows that do not have an STIG ID defined in the STIG ID and have a status of “Applicable - Configurable”.
Delta: If a rule is not the same an HTML diff will appear; base content is on the left.

Example:

    $ ./utils/srg_diff.py --target submission.xlsx --base build/cac_stig_output.xlsx --output build/diff.html -p rhel9
    Wrote output to build/diff.html.

`utils/shorthand_to_oval.py` – Convert shorthand OVAL to a full OVAL

This script converts (resolved) shorthand OVAL files to a valid OVAL file.

It can be useful for example when creating minimal bug reproducers. If you know that a problem is located in OVAL for a specific rule, you can use it to convert a resolved shorthand OVAL files from the build/product/checks/oval directory to a standalone valid OVAL file that you can then pass to oscap.

Example:

$ ./build_product rhel9
$ utils/shorthand_to_oval.py build/rhel9/checks/oval/accounts_tmout.xml oval.xml
$ oscap oval eval oval.xml

`utils/controlrefcheck.py` – Ensure Control Files and Rules Are Consistent

This script helps ensure that control files and rules files are in sync. The script takes in what product, control, and reference you are looking for. The script loops over every rule in each control; the script then checks if the control’s id is in the rule’s reference that passed to the script. If a control’s ID does not match or rule does not exist in the project the script will exit with a non-zero status. If the reference you are looking for is cis the script will not attempt to match anything that does not match a CIS section number.

To execute:

$ ./utils/controlrefcheck.py rhel9 cis_rhel9 cis

`utils/check_eof.py` - Ensure Files Follow EOF Style

This script checks files of specific extensions (see EXTENSIONS in the script for the full list) to ensure that the files end with new line as required by the style guide. This script can be used to fix files that don’t have a newline at the end if the --fix flag is passed to the script. By default, the script just outputs a list of files are non-compliant.

To execute:

$ ./utils/check_eof.py ssg linux_os utils tests products shared docs apple_os applications build-scripts cmake Dockerfiles

`utils/generate_profile.py` – Generating CIS Control Files

This script accepts a CIS benchmark spreadsheet (XLSX) and outputs a profile, section, or rule. This is primarily useful for contributors maintaining content. The script doesn’t make assumptions about rules that implement controls, should still be done by someone knowledge about the platform and benchmark. The goal of the script is to reduce the amount of text contributors have to copy and paste from benchmarks, making it easier to automate parts of the benchmark maintenance process.

You can download CIS XLSX spreadsheets from CIS directly if you have access to CIS workbench.

You can use the script to list all controls in a benchmark:

$ python utils/generate_profile.py -i benchmark.xlsx list
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
1.10
...

To generate a rule for a specific control:

$ python utils/generate_profile.py -i benchmark.xlsx generate -c 1.1.2
documentation_complete: false
title: |-
  Ensure that the API server pod specification file ownership is set to root:root
description: 'Ensure that the API server pod specification file ownership is set to
  `root:root`.

  No remediation required; file permissions are managed by the operator.'
rationale: |-
  The API server pod specification file controls various parameters that set the behavior of the API server. You should set its file ownership to maintain the integrity of the file. The file should be owned by `root:root`.
severity: PLACEHOLDER
references: PLACEHOLDER
ocil: "OpenShift 4 deploys two API servers: the OpenShift API server and the Kube\
  \ API server. \n\nThe OpenShift API server is managed as a deployment. The pod specification\
  \ yaml for openshift-apiserver is stored in etcd. \n\nThe Kube API Server is managed\
  \ as a static pod. The pod specification file for the kube-apiserver is created\
  \ on the control plane nodes at /etc/kubernetes/manifests/kube-apiserver-pod.yaml.\
  \ The kube-apiserver is mounted via hostpath to the kube-apiserver pods via /etc/kubernetes/static-pod-resources/kube-apiserver-pod.yaml\
  \ with ownership `root:root`.\n\nTo verify pod specification file ownership for\
  \ the kube-apiserver, run the following command.\n\n```\n#echo \u201Ccheck kube-apiserver\
  \ pod specification file ownership\u201D\n\nfor i in $( oc get pods -n openshift-kube-apiserver\
  \ -l app=openshift-kube-apiserver -o name )\ndo\n oc exec -n openshift-kube-apiserver\
  \ $i -- \\\n stat -c %U:%G /etc/kubernetes/static-pod-resources/kube-apiserver-pod.yaml\n\
  done\n```\nVerify that the ownership is set to `root:root`."
ocil_clause: PLACEHOLDER
warnings: PLACEHOLDER
template: PLACEHOLDER

To generate an entire section:

$ python utils/generate_profile.py -i benchmark.xlsx generate -s 1

The PLACEHOLDER values must be filled in later, ideally when the rules are provided for each control.

`utils/compare_versions.py` – Compare ComplianceAsCode versions

Show differences between two ComplianceAsCode versions. Lists added or removed rules, profiles, changes in profile composition and changes in remediations and platforms. For comparison, you can use git tags or ComplianceAsCode JSON manifest files directly.

To compare 2 ComplianceAsCode JSON manifests, provide the manifest files.

python3 utils/compare_versions.py compare_manifests ~/manifests/old.json ~/manifests/new.json

To compare 2 upstream versions, you need to specify the version git tags and a product ID.

$ python3 utils/compare_versions.py compare_tags v0.1.67 v0.1.68 rhel9

It will internally clone the upstream project, checkout these tags, generate ComplianceAsCode JSON manifests, compare them and print the output.

`utils/oscal/build_cd_from_policy.py` – Build a Component Definition from a Policy

This script builds an OSCAL Component Definition (cd) (version 1.0.4) for an existing OSCAL profile from a policy. The script uses the compliance-trestle library to build the component definition. The component definition can be used with the compliance-trestle CLI after generation.

Some assumption made by this script:

The script maps control file statuses to valid OSCAL statuses as follows:
- pending - alternative
- not applicable: not-applicable
- inherently met: implemented
- documentation: implemented
- planned: planned
- partial: partial
- supported: implemented
- automated: implemented
- manual: alternative
- does not meet: alternative
The script uses the “Section letter:” convention in the control notes to create statements under the implemented requirements.
The script maps parameter to rules uses the xccdf_variable field under template.vars
To determine what responses will mapped to the controls in the OSCAL profile the control id and label property from the resolved catalog is searched.

It supports the following arguments:

-o, --output — Path to write the cd to
-r, --root — Root of the SSG project. Defaults to /content.
-v, --vendor-dir — Path to the vendor directory with third party OSCAL artifacts
-p, --profile — Main profile href, or name of the profile model in the trestle workspace
-pr, --product — Product to build cd with
-c, --control — Control to use as the source for control responses. To optionally filter by level, use the format <control_id>:.
-j, --json — Path to the rules_dir.json. Defaults to /content/build/rule_dirs.json.
-b, --build-config-yaml — YAML file with information about the build configuration
-t, --component-definition-type — Type of component definition to create. Defaults to service. Options are service or validation.

An example of how to execute the script:

$ ./build_product ocp4
$ ./utils/rule_dir_json.py
$ ./utils/oscal/build_cd_from_policy.py -o build/ocp4.json -p fedramp_rev4_high -pr ocp4 -c nist_ocp4:high

`utils/ansible_playbook_to_role.py` – Generates Ansible Roles and pushes them to Github

This script converts the Ansible playbooks created by the build system and converts them to Ansible roles and can upload them to GitHub.

An example of how to execute the script to generate roles locally:

$ ./build_product rhel9
$ ./utils/ansible_playbook_to_role.py --dry-run output

`utils/find_unused_rules.py` – List Rules That Are Not Used In Any Data stream

This script will output rules are not in any data streams. To prevent false positives the script will not run if the number of build datas treams less than the total number of products in the project. The script assumes that ./build_project --derivatives was executed before the script is used. This script does require that ./utils/rule_dir_json.py was executed before this script is used as well.

This script works by comparing rules in the data streams to the rules in the rule_dirs.json file. The script works by adding off the rule ids from the data streams to a set. Then the script converts the keys of rule_dirs.json to a set. The set of rules in the data stream is subtracted to from the set of rules in rule_dirs.json. The difference is then output to the user.

Example usage:

$ ./build_product --derivatives
$ ./utils/rule_dir_json.py
$ ./utils/find_unused_rules.py

Tools and Utilities

Profile Statistics and Utilities

Generating Controls from DISA’s XCCDF Files

Generating Controls From a Reference

Generating login banner regular expressions

Modifying rule directory content files

utils/fix_rules.py – automatically fix-up rules

How to automatically assign CCEs with the add-cce sub-command

utils/refchecker.py – automatically check rule.yml for references

utils/mod_checks.py and utils/mod_fixes.py – programmatically modify check and fix applicability

utils/create_scap_delta_tailoring.py – Create tailoring files for rules not covered by other content

utils/compare_ds.py – Compare two data streams (can also compare XCCDFs)

HTML Diffs

utils/compare_results.py – Compare to two ARF result files

utils/import_srg_spreadsheet.py – Import changes made to an SRG Spreadsheet into the project

utils/import_disa_stig.py – Import Content from DISA’s XML Content

Profiling the Build System

utils/build_profiler.sh – Handle directory structure for profiling files and invokes other script

utils/build_profiler_report.py – Parse a ninja file and display report to user

Other Scripts

utils/srg_diff.py – Compare Two SRG Spreadsheets

utils/shorthand_to_oval.py – Convert shorthand OVAL to a full OVAL

utils/controlrefcheck.py – Ensure Control Files and Rules Are Consistent

utils/check_eof.py - Ensure Files Follow EOF Style

utils/generate_profile.py – Generating CIS Control Files

utils/compare_versions.py – Compare ComplianceAsCode versions

utils/oscal/build_cd_from_policy.py – Build a Component Definition from a Policy

utils/ansible_playbook_to_role.py – Generates Ansible Roles and pushes them to Github

utils/find_unused_rules.py – List Rules That Are Not Used In Any Data stream

`utils/fix_rules.py` – automatically fix-up rules

How to automatically assign CCEs with the `add-cce` sub-command

`utils/refchecker.py` – automatically check `rule.yml` for references

`utils/mod_checks.py` and `utils/mod_fixes.py` – programmatically modify check and fix applicability

`utils/create_scap_delta_tailoring.py` – Create tailoring files for rules not covered by other content

`utils/compare_ds.py` – Compare two data streams (can also compare XCCDFs)

`utils/compare_results.py` – Compare to two ARF result files

`utils/import_srg_spreadsheet.py` – Import changes made to an SRG Spreadsheet into the project

`utils/import_disa_stig.py` – Import Content from DISA’s XML Content

`utils/build_profiler.sh` – Handle directory structure for profiling files and invokes other script

`utils/build_profiler_report.py` – Parse a ninja file and display report to user

`utils/srg_diff.py` – Compare Two SRG Spreadsheets

`utils/shorthand_to_oval.py` – Convert shorthand OVAL to a full OVAL

`utils/controlrefcheck.py` – Ensure Control Files and Rules Are Consistent

`utils/check_eof.py` - Ensure Files Follow EOF Style

`utils/generate_profile.py` – Generating CIS Control Files

`utils/compare_versions.py` – Compare ComplianceAsCode versions

`utils/oscal/build_cd_from_policy.py` – Build a Component Definition from a Policy

`utils/ansible_playbook_to_role.py` – Generates Ansible Roles and pushes them to Github

`utils/find_unused_rules.py` – List Rules That Are Not Used In Any Data stream