ssg package

ssg.ansible module

Common functions for processing Ansible in SSG

class ssg.ansible.AnsibleSnippetsProcessor(all_snippets)

Bases: object

Processes Ansible snippets to optimize package management tasks.

This class processes a collection of Ansible snippets by: - Collecting package-related tasks for batch processing - Skipping redundant package_facts tasks - Handling block tasks recursively - Preserving other tasks as-is

get_ansible_tasks()

Get the final list of processed Ansible tasks.

Package facts tasks are added at the beginning and end of package tasks, then combined with other tasks.

Returns:

Combined list of all processed tasks.

Return type:

list

process_snippets()

Process all snippets provided during initialization.

ssg.ansible.add_minimum_version(ansible_src)

Adds a minimum Ansible version requirement to an Ansible script.

This function inserts a pre_task into the provided Ansible script to assert that the Ansible version is greater than or equal to a specified minimum version. If the script already contains a pre_task or the version check, it will return the original script. If a pre_task exists but does not contain the version check, it raises a ValueError.

Parameters:

ansible_src (str) – The source code of the Ansible script.

Returns:

The modified Ansible script with the minimum version requirement added.

Return type:

str

Raises:

ValueError – If a pre_task already exists in the Ansible script but does not contain the version check.

ssg.ansible.remove_too_many_blank_lines(ansible_src)

Condenses three or more consecutive empty lines into two empty lines.

Parameters:

ansible_src (str) – The source string from an Ansible file.

Returns:

The modified string with excessive blank lines reduced.

Return type:

str

ssg.ansible.remove_trailing_whitespace(ansible_src)

Remove trailing whitespace from each line in the given Ansible source string.

Parameters:

ansible_src (str) – The Ansible source code as a string.

Returns:

The Ansible source code with trailing whitespace removed from each line.

Return type:

str

ssg.ansible.task_is(task, names)

Check if the task is one of the given names.

Parameters:

• task (dict) – The task to check.

• names (list) – List of task names to check against.

Returns:

True if the task contains any of the given names as keys, False otherwise.

Return type:

bool

ssg.boolean_expression module

Common functions for Boolean Expressions

class ssg.boolean_expression.Algebra(symbol_cls, function_cls)

Bases: BooleanAlgebra

Base class for boolean algebra.

The Algebra class parses and evaluates boolean expressions, where operators can be any combination of “~, &, |, !, *, +, not, and, or” and variable symbols can contain version specifiers as described in PEP440 and PEP508. - No white space is allowed inside specifier expressions. - The ~= specifier operator is not supported.

Limitations: - no white space is allowed inside specifier expressions; - ~= specifier operator is not supported.

Example

“(oranges>=2.0.8,<=5 | fried[banana]) and !pie[apple]”

symbol_cls

The class used for symbols in the boolean expressions.

Type:

class

function_cls

The class used for functions in the boolean expressions.

Type:

class

class ssg.boolean_expression.Function(*args)

Bases: Function

Base class for boolean functions.

This class should be subclassed and passed to the Algebra as function_cls to enrich expression elements with domain-specific methods.

as_id()

Generate a string representation of the boolean expression.

This method constructs a unique identifier for the boolean expression by recursively calling as_id on its arguments and combining them with the appropriate boolean operator.

Returns:

A string representing the boolean expression. If the expression is a negation, it

returns ‘not_’ followed by the identifier of the negated argument. If the expression is a conjunction or disjunction, it returns the identifiers of the arguments joined by ‘_and_’ or ‘_or_’.

Return type:

str

is_and()

is_not()

is_or()

class ssg.boolean_expression.Symbol(obj)

Bases: Symbol

Symbol class represents a boolean symbol with domain-specific methods.

This class should be subclassed and passed to the Algebra as symbol_cls to enrich expression elements with domain-specific methods.

requirement

The requirement object associated with the symbol.

Type:

requirement_specs.Requirement

obj

Alias for the requirement attribute.

Type:

requirement_specs.Requirement

property arg

as_dict()

as_id()

static get_base_of_parametrized_name(name)

has_version_specs()

static is_parametrized(name)

property name

ssg.build_cpe module

Common functions for building CPEs

class ssg.build_cpe.CPEALCheckFactRef(obj)

Bases: Symbol

static cpe_id_is_parametrized(cpe_id)

enrich_with_cpe_info(cpe_products)

static get_base_name_of_parametrized_cpe_id(cpe_id)

If given a parametrized platform name such as package[test], it returns the package part only.

ns = 'http://cpe.mitre.org/language/2.0'

prefix = 'cpe-lang'

to_ansible_conditional()

to_bash_conditional()

to_xml_element()

class ssg.build_cpe.CPEALLogicalTest(*args)

Bases: Function

enrich_with_cpe_info(cpe_products)

ns = 'http://cpe.mitre.org/language/2.0'

prefix = 'cpe-lang'

to_ansible_conditional()

to_bash_conditional()

to_xml_element()

exception ssg.build_cpe.CPEDoesNotExist

Bases: Exception

class ssg.build_cpe.CPEItem(id_)

Bases: XCCDFEntity, Templatable

Represents the cpe-item element from the CPE standard.

KEYS: Dict[str, Callable[[], Any | None]] = {'ansible_conditional': <function CPEItem.<lambda>>, 'args': <function CPEItem.<lambda>>, 'bash_conditional': <function CPEItem.<lambda>>, 'check_id': <function CPEItem.<lambda>>, 'content_id': <function CPEItem.<lambda>>, 'definition_location': <function XCCDFEntity.<lambda>>, 'id_': <function XCCDFEntity.<lambda>>, 'is_product_cpe': <function CPEItem.<lambda>>, 'name': <function CPEItem.<lambda>>, 'template': <function Templatable.<lambda>>, 'title': <function XCCDFEntity.<lambda>>, 'versioned': <function CPEItem.<lambda>>}

MANDATORY_KEYS: Set[str] = {'name'}

property cpe_oval_def_id

property cpe_oval_short_def_id

create_resolved_cpe_item_for_fact_ref(fact_ref)

classmethod from_yaml(yaml_file, env_yaml=None, product_cpes=None)

static is_cpe_name(cpe_id_or_name)

ns = 'http://cpe.mitre.org/dictionary/2.0'

prefix = 'cpe-dict'

set_conditional(language, content)

set_cpe_oval_def_id()

set_template_variables(*sources)

to_xml_element(cpe_oval_filename)

class ssg.build_cpe.CPEList

Bases: object

Represents the cpe-list element from the CPE standard.

add(cpe_item)

ns = 'http://cpe.mitre.org/dictionary/2.0'

prefix = 'cpe-dict'

to_file(file_name, cpe_oval_file, selection_of_cpe_names=None)

to_xml_element(cpe_oval_file, selection_of_cpe_names=None)

translate_cpe_oval_def_ids()

class ssg.build_cpe.ProductCPEs

Bases: object

Reads from the disk all the yaml CPEs related to a product and provides them in a structured way.

add_cpe_item(cpe_item)

add_resolved_cpe_items_from_platform(platform)

get_cpe(cpe_id_or_name)

get_cpe_for_fact_ref(fact_ref)

get_cpe_name(cpe_id)

get_product_cpe_names()

load_content_cpes(env_yaml)

load_cpes_from_directory_tree(root_path, env_yaml)

load_cpes_from_list(cpes_list)

load_product_cpes(env_yaml)

load_product_cpes_from_list(product_cpes_list)

ssg.build_cpe.extract_env_obj(objects, local_var)

From a collection of objects, return the object with id matching the object_ref of the local variable.

NOTE: This assumes that a local variable can only reference one object. Which is not true, variables can reference multiple objects. But this assumption should work for OVAL checks for CPEs, as they are not that complicated.

ssg.build_cpe.extract_referred_nodes(tree_with_refs, tree_with_ids, attrname)

Return the elements in tree_with_ids which are referenced from tree_with_refs via the element attribute ‘attrname’.

ssg.build_cpe.extract_subelement(objects, sub_elem_type)

From a collection of element objects, return the value of the first attribute of name sub_elem_type found.

This is useful when the object is a single element and we wish to query some external reference identifier in the subtree of that element.

ssg.build_cpe.get_linked_cpe_oval_document(unlinked_oval_file_path)

ssg.build_derivatives module

Common functions for enabling derivative products

ssg.build_derivatives.add_cpe_item_to_dictionary(tree_root, product_yaml_path, cpe_ref, id_name, cpe_items_dir)

ssg.build_derivatives.add_cpes(elem, namespace, mapping)

Adds derivative CPEs next to RHEL ones, checks XCCDF elements of given namespace.

ssg.build_derivatives.add_element_to(oval_root, tag_name, component_element)

ssg.build_derivatives.add_notice(benchmark, namespace, notice, warning)

Adds derivative notice as the first notice to given benchmark.

ssg.build_derivatives.add_oval_components_to_oval_xml(oval_root, tag_name, component_dict)

ssg.build_derivatives.add_oval_definition_to_cpe_oval(root, unlinked_oval_file_path, oval_def_id)

ssg.build_derivatives.get_cpe_item(product_yaml, cpe_ref, cpe_items_dir)

ssg.build_derivatives.get_cpe_oval_root(root)

ssg.build_derivatives.profile_handling(tree_root, namespace)

ssg.build_derivatives.remove_cce_reference(tree_root, namespace)

Remove CCE identifiers from OVAL checks in XML tree

ssg.build_derivatives.remove_idents(tree_root, namespace, prod='RHEL')

Remove product identifiers from rules in XML tree

ssg.build_derivatives.replace_platform(tree_root, namespace, product)

ssg.build_guides module

ssg.build_guides.build_index(benchmarks, input_basename, index_links, index_options, index_initial_src)

ssg.build_guides.builder(task_queue)

Fetch from a queue of tasks, process tasks until the queue is empty. Each task is processed with generate_for_input_content, and the guide is written as output.

Raises: when an error occurred when processing a task.

ssg.build_guides.fill_queue(benchmarks, benchmark_profile_pairs, input_path, path_base, output_dir)

For each benchmark and profile in the benchmark, create a queue of tasks for later processing. A task is a named tuple (benchmark_id, profile_id, input_path, guide_path).

Returns: queue of tasks.

ssg.build_guides.generate_for_input_content(input_content, benchmark_id, profile_id)

Returns HTML guide for given input_content and profile_id combination. This function assumes only one Benchmark exists in given input_content!

ssg.build_guides.get_benchmark_profile_pairs(input_tree, benchmarks)

ssg.build_guides.get_output_guide_paths(benchmarks, benchmark_profile_pairs, path_base, output_dir)

Return a list of guide paths containing guides for each non-skipped profile_id in a benchmark.

ssg.build_guides.get_path_args(args)

Return a namedtuple of (input_path, input_basename, path_base, output_dir) from an argparse containing args.input and args.output.

ssg.build_ovals module

exception ssg.build_ovals.OVALBuildException

Bases: Exception

class ssg.build_ovals.OVALBuilder(env_yaml, product_yaml_path, shared_directories, build_ovals_dir)

Bases: object

get_oval_document_from_shorthands(include_benchmark)

property product_name

ssg.build_ovals.expand_shorthand(shorthand_path, oval_path, env_yaml)

ssg.build_profile module

class ssg.build_profile.RuleStats(rule, cis_ns)

Bases: object

Class representing the content of a rule for statistics generation purposes.

class ssg.build_profile.XCCDFBenchmark(filepath, product='')

Bases: object

Class for processing an XCCDF benchmark to generate statistics about the profiles contained within it.

console_print(content, width)

Prints the ‘content’ array left aligned, each time 45 characters long, each row ‘width’ characters wide

get_all_profile_stats()

get_profile_stats(profile)

Obtain statistics for the profile

show_all_profile_stats(options)

show_profile_stats(profile, options)

Displays statistics for specific profile

ssg.build_profile.get_cis_uri(benchmark_tree)

ssg.build_profile.make_name_to_profile_mapping(profile_files, env_yaml, product_cpes)

ssg.build_remediations module

class ssg.build_remediations.AnacondaRemediation(file_path)

Bases: Remediation

class ssg.build_remediations.AnsibleRemediation(file_path)

Bases: Remediation

classmethod from_snippet_and_rule(snippet_fname, rule_fname)

get_references()

inject_package_facts_task(parsed_snippet)

Injects a package_facts task only if the snippet has a task with a when clause with ansible_facts.packages, and the snippet doesn’t already have a package_facts task

parse_from_file(env_yaml, cpe_platforms)

update(parsed, config, cpe_platforms)

update_tags_from_config(to_update, config)

update_tags_from_rule(to_update)

update_when_from_rule(to_update, cpe_platforms)

class ssg.build_remediations.BashRemediation(file_path)

Bases: Remediation

parse_from_file(env_yaml, cpe_platforms)

static wrap_conditionals_with_operators(conditionals)

Wrap bash conditionals that contain operators to ensure proper short-circuit evaluation.

When multiple platform conditionals are joined with OR (||), each conditional that contains operators (&& or ||) must be wrapped in parentheses to ensure proper bash short-circuit evaluation.

Without proper wrapping:

grep … && { version_check } || grep … && { version_check }

causes all version checks to execute due to bash operator precedence.

With proper wrapping:

( grep … && { version_check } ) || ( grep … && { version_check } )

ensures only the matching platform’s version check executes.

Parameters:

conditionals – List of bash conditional expressions

Returns:

List of conditionals with those containing operators wrapped in parentheses

class ssg.build_remediations.BlueprintRemediation(file_path)

Bases: Remediation

This provides class for OSBuild Blueprint remediations

class ssg.build_remediations.BootcRemediation(file_path)

Bases: Remediation

This provides class for Bootc remediations

class ssg.build_remediations.HummingbirdRemediation(file_path)

Bases: Remediation

This provides class for Hummingbird remediations

class ssg.build_remediations.IgnitionRemediation(file_path)

Bases: Remediation

class ssg.build_remediations.KickstartRemediation(file_path)

Bases: Remediation

This provides class for Kickstart remediations

class ssg.build_remediations.KubernetesRemediation(file_path)

Bases: Remediation

class ssg.build_remediations.PuppetRemediation(file_path)

Bases: Remediation

class ssg.build_remediations.Remediation(file_path, remediation_type)

Bases: object

associate_rule(rule_obj)

expand_env_yaml_from_rule()

get_inherited_conditionals(language, cpe_platforms)

get_inherited_cpe_platform_names()

get_rule_specific_conditionals(language, cpe_platforms)

get_rule_specific_cpe_platform_names()

get_stripped_conditionals(language, cpe_platform_names, cpe_platforms)

collect conditionals of platforms defined by cpe_platform_names and strip them of white spaces

parse_from_file(env_yaml, cpe_platforms)

class ssg.build_remediations.RemediationObject(contents, config)

Bases: tuple

config

Alias for field number 1

contents

Alias for field number 0

ssg.build_remediations.expand_xccdf_subs(fix, remediation_type)

Expand the respective populate keywords of each remediation type with an <xccdf:sub> element

This routine translates any instance of the ‘type-populate’ keyword in the form of:

(type-populate variable_name)

where type can be either ansible, puppet, anaconda or bash, into

<sub idref=”variable_name”/>

ssg.build_remediations.get_rule_dir_remediations(dir_path, remediation_type, product=None)

Gets a list of remediations of type remediation_type contained in a rule directory. If product is None, returns all such remediations. If product is not None, returns applicable remediations in order of priority:

{{{ product }}}.ext -> shared.ext

Only returns remediations which exist.

ssg.build_remediations.is_supported_filename(remediation_type, filename)

Checks if filename has a supported extension for remediation_type.

Exits when remediation_type is of an unknown type.

ssg.build_remediations.load_compiled_remediations(fixes_dir)

ssg.build_remediations.parse_from_file_with_jinja(file_path, env_yaml)

Parses a remediation from a file. As remediations contain jinja macros, we need a env_yaml context to process these. In practice, no remediations use jinja in the configuration, so for extracting only the configuration, env_yaml can be an abritrary product.yml dictionary.

If the logic of configuration parsing changes significantly, please also update ssg.fixes.parse_platform(…).

ssg.build_remediations.parse_from_file_without_jinja(file_path)

Parses a remediation from a file. Doesn’t process the Jinja macros. This function is useful in build phases in which all the Jinja macros are already resolved.

ssg.build_remediations.process(remediation, env_yaml, cpe_platforms)

Process a fix, and return the processed fix iff the file is of a valid extension for the remediation type and the fix is valid for the current product.

Note that platform is a required field in the contents of the fix.

ssg.build_remediations.split_remediation_content_and_metadata(fix_file)

ssg.build_remediations.write_fix_to_file(fix, file_path)

Writes a single fix to the given file path.

ssg.build_renumber module

class ssg.build_renumber.FileLinker(translator, xccdftree, checks, output_file_name)

Bases: object

Bass class which represents the linking of checks to their identifiers.

CHECK_NAMESPACE: str | None = None

CHECK_SYSTEM: str | None = None

add_missing_check_exports(check, checkcontentref)

get_related_checks(checks)

Returns a list of checks which have the same check system as this class.

link_xccdf()

save_linked_tree()

Write internal tree to the file in self.linked_fname.

class ssg.build_renumber.OCILFileLinker(translator, xccdftree, checks, output_file_name)

Bases: FileLinker

CHECK_NAMESPACE: str | None = 'http://scap.nist.gov/schema/ocil/2.0'

CHECK_SYSTEM: str | None = 'http://scap.nist.gov/schema/ocil/2'

link(tree)

class ssg.build_renumber.OVALFileLinker(translator, xccdftree, checks, output_file_name)

Bases: FileLinker

CHECK_NAMESPACE: str | None = 'http://oval.mitre.org/XMLSchema/oval-definitions-5'

CHECK_SYSTEM: str | None = 'http://oval.mitre.org/XMLSchema/oval-definitions-5'

add_missing_check_exports(check, checkcontentref)

build_ovals_dir = None

link()

save_linked_tree()

Write internal tree to the file in self.linked_fname.

save_oval_document_for_each_xccdf_rule(file_name_prefix='')

ssg.build_renumber.check_and_correct_xccdf_to_oval_data_export_matching_constraints(xccdftree, oval_document)

Verify if <xccdf:Value> ‘type’ to corresponding OVAL variable ‘datatype’ export matching constraint:

http://csrc.nist.gov/publications/nistpubs/800-126-rev2/SP800-126r2.pdf#page=30&zoom=auto,69,313

is met. Also correct the ‘type’ attribute of those <xccdf:Value> elements where necessary in order the produced content to meet this constraint.

To correct the constraint we use simpler approach - prefer to fix ‘type’ attribute of <xccdf:Value> rather than ‘datatype’ attribute of the corresponding OVAL variable since there might be additional OVAL variables, derived from the affected OVAL variable, and in that case we would need to fix the ‘datatype’ attribute in each of them.

Define the <xccdf:Value> ‘type’ to OVAL variable ‘datatype’ export matching constraints mapping as specified in Table 16 of XCCDF v1.2 standard:

http://csrc.nist.gov/publications/nistpubs/800-126-rev2/SP800-126r2.pdf#page=30&zoom=auto,69,313

ssg.build_renumber.create_xccdf_id_to_cce_id_mapping(xccdftree)

ssg.build_renumber.rules_with_ids_generator(xccdftree)

ssg.build_renumber.verify_correct_form_of_referenced_cce_identifiers(xccdftree)

In SSG benchmarks, the CCEs till unassigned have the form of e.g. “RHEL7-CCE-TBD” (or any other format possibly not matching the above two requirements)

If this is the case for specific SSG product, drop such CCE identifiers from the XCCDF since they are in invalid format!

ssg.build_sce module

class ssg.build_sce.SCEBuilder(env_yaml, product_yaml, template_builder, output_dir)

Bases: object

build()

Walks the content repository and builds all SCE checks (and metadata entry) into the output directory.

ssg.build_sce.collect_sce_checks(datastreamtree)

ssg.build_sce.load_sce_and_metadata(file_path, local_env_yaml)

For the given SCE audit file (file_path) under the specified environment (local_env_yaml), parse the file while expanding Jinja macros and read any metadata headers the file contains. Note that the last keyword of a specified type is the recorded one.

Parameters:

• file_path (str) – The path to the SCE audit file

• local_env_yaml (dict) – The local environment dict

Returns (audit_content, metadata).

ssg.build_sce.load_sce_and_metadata_parsed(raw_content)

ssg.build_sce.write_sce_file(sce_content, output_dir, filename)

ssg.build_stig module

ssg.build_stig.get_description_root(srg)

ssg.build_stig.get_severity(input_severity)

ssg.build_stig.map_versions_to_rule_ids(reference_file_name)

ssg.build_stig.parse_srgs(xml_path)

ssg.build_yaml module

Common functions for building YAML in SSG. Also contains definitions of basic classes like Rule, Group, Value and Platform.

class ssg.build_yaml.Benchmark(id_)

Bases: XCCDFEntity

Represents an XCCDF Benchmark entity with various attributes and methods to manipulate and represent the benchmark data.

KEYS

Dictionary of keys with default values.

Type:

dict

MANDATORY_KEYS

Set of mandatory keys for the benchmark.

Type:

set

GENERIC_FILENAME

Default filename for the benchmark.

Type:

str

GENERIC_FILENAME = 'benchmark.yml'

KEYS: Dict[str, Callable[[], Any | None]] = {'cpes': <function Benchmark.<lambda>>, 'definition_location': <function XCCDFEntity.<lambda>>, 'description': <function Benchmark.<lambda>>, 'front_matter': <function Benchmark.<lambda>>, 'groups': <function Benchmark.<lambda>>, 'id_': <function XCCDFEntity.<lambda>>, 'notice_description': <function Benchmark.<lambda>>, 'notice_id': <function Benchmark.<lambda>>, 'platforms': <function Benchmark.<lambda>>, 'product_cpe_names': <function Benchmark.<lambda>>, 'profiles': <function Benchmark.<lambda>>, 'rear_matter': <function Benchmark.<lambda>>, 'rules': <function Benchmark.<lambda>>, 'status': <function Benchmark.<lambda>>, 'title': <function XCCDFEntity.<lambda>>, 'values': <function Benchmark.<lambda>>, 'version': <function Benchmark.<lambda>>}

MANDATORY_KEYS: Set[str] = {'description', 'front_matter', 'rear_matter', 'status', 'title'}

add_group(group, env_yaml=None, product_cpes=None)

Adds a group to the groups dictionary.

Parameters:

• group (Group) – The group object to be added. Must have an ‘id_’ attribute.

• env_yaml (dict, optional) – Additional environment YAML data. Default is None.

• product_cpes (dict, optional) – Additional product CPEs data. Default is None.

Returns:

None

add_profiles_from_dir(dir_, env_yaml, product_cpes)

Adds profiles from the specified directory to the current instance.

This method scans the given directory for files with the ‘.profile’ extension, attempts to create ProfileWithInlinePolicies objects from them, and appends them to the instance’s profiles list.

Parameters:

• dir (str) – The directory to scan for profile files.

• env_yaml (dict) – The environment YAML data used for profile creation.

• product_cpes (list) – The list of product CPEs used for profile creation.

Returns:

None

Raises:

RuntimeError – If there is an error building a profile from a file.

Notes

• Files that do not have the ‘.profile’ extension are skipped.

• If a profile file is incomplete or an error occurs during its creation, it is skipped.

add_rule(rule)

Adds a rule to the rules dictionary.

Parameters:

rule (Rule) – The rule to be added. If None, the method returns without adding anything.

Returns:

None

add_value(value)

Adds a value to the values dictionary if the value is not None.

Parameters:

value (object) – The value to be added. It is expected to have an ‘id_’ attribute.

Returns:

None

drop_rules_not_included_in_a_profile()

Removes rules from groups that are not included in any profile.

This method retrieves the set of rules that are selected in all profiles and removes any rules from each group that are not listed in this set.

Returns:

None

classmethod from_yaml(yaml_file, env_yaml=None, product_cpes=None)

Creates a Benchmark instance from a YAML file.

Parameters:

• yaml_file (str) – Path to the YAML file.

• env_yaml (dict, optional) – Environment-specific YAML data. Defaults to None.

• product_cpes (ProductCPEs, optional) – Product CPEs instance. Defaults to None.

Returns:

An instance of the Benchmark class populated with data from the YAML file.

Return type:

Benchmark

get_benchmark_xml_for_profiles(env_yaml, profiles, rule_and_variables_dict, include_contributors=True)

Generates the benchmark XML for the given profiles.

Parameters:

• env_yaml (dict) – The environment YAML configuration.

• profiles (list) – A list of profile objects.

• rule_and_variables_dict (dict) – A dictionary containing rules and variables.

Returns:

A tuple containing:

• profiles_ids (list): A list of profile IDs.

• xml_element (Element): The XML element representing the benchmark.

Return type:

tuple

get_components_not_included_in_a_profiles(profiles, rules_and_variables_dict)

Identify and return the sets of rules, groups, and variables that are not included in any of the given profiles.

Parameters:

• profiles (list) – A list of profiles to check against.

• rules_and_variables_dict (dict) – A dictionary containing rules and their associated variables.

Returns:

A tuple containing three sets:

• rules (set): A set of rules not included in any of the profiles.

• groups (set): A set of groups not included in any of the profiles.

• variables (set): A set of variables not included in any of the profiles.

Return type:

tuple

get_not_used_cpe_platforms(profiles)

Get the CPE platforms that are not used in the given profiles.

Parameters:

profiles (list) – A list of profiles to check for used CPE platforms.

Returns:

A set of CPE platforms that are not used in the given profiles.

Return type:

set

get_rules_selected_in_all_profiles(profiles=None)

Get the set of rules that are selected in all given profiles.

Parameters:

profiles (list, optional) – A list of profile objects. If None, the method will use the instance’s profiles attribute.

Returns:

A set of rules that are selected in all provided profiles.

Return type:

set

get_used_cpe_platforms(profiles)

Retrieves the CPE platforms used by the selected rules in the given profiles.

Parameters:

profiles (list) – A list of profiles to check for selected rules.

Returns:

A list of CPE platforms associated with the selected rules.

Return type:

list

static get_variables_of_rules(profiles, rule_ids, rules_and_variables_dict)

Collects and returns a set of variables associated with the given rules and profiles.

Parameters:

• profiles (list) – A list of profile objects, each containing a dictionary of variables.

• rule_ids (list) – A list of rule identifiers.

• rules_and_variables_dict (dict) – A dictionary mapping rule identifiers to sets of variables.

Returns:

A set of variables associated with the specified rules and profiles.

Return type:

set

load_entities(rules_by_id, values_by_id, groups_by_id)

Load entities into the current object from provided dictionaries if they are not already set.

Parameters:

• rules_by_id (dict) – A dictionary containing rule entities indexed by their IDs.

• values_by_id (dict) – A dictionary containing value entities indexed by their IDs.

• groups_by_id (dict) – A dictionary containing group entities indexed by their IDs.

This method updates the rules, values, and groups attributes of the current object. If an entity in these attributes is not already set (i.e., its value is falsy), it will be loaded from the corresponding provided dictionary.

classmethod process_input_dict(input_contents, env_yaml, product_cpes)

Processes the input dictionary by transforming specific keys and extracting required data.

Parameters:

• cls (type) – The class that calls this method.

• input_contents (dict) – The dictionary containing the input data to be processed.

• env_yaml (dict) – The environment YAML data.

• product_cpes (list) – The list of product CPEs.

Returns:

The processed data dictionary with transformed and extracted information.

Return type:

dict

Raises:

KeyError – If any required key is missing in the input dictionaries.

represent_as_dict()

Converts the instance attributes to a dictionary representation, modifying specific keys for compatibility.

Returns:

A dictionary representation of the instance with modified keys.

Return type:

dict

to_file(file_name, env_yaml=None)

Serializes the XML representation of the object to a file.

Parameters:

• file_name (str) – The name of the file to which the XML data will be written.

• env_yaml (dict, optional) – An optional parameter that can be used to customize the XML generation.

Returns:

None

to_xccdf()

Converts the current object to XCCDF format.

This method is intended to be extended to generate a valid XCCDF instead of SSG SHORTHAND.

Returns:

None

Raises:

NotImplementedError – This method is not yet implemented.

to_xml_element(env_yaml=None, product_cpes=None, components_to_not_include=None, include_contributors=True)

Converts the current object to an XML element.

Parameters:

• env_yaml (dict, optional) – Environment YAML data. Defaults to None.

• product_cpes (list, optional) – List of product CPEs. Defaults to None.

• components_to_not_include (dict, optional) – Components to exclude from the XML. Defaults to None.

• include_contributors (bool, optional) – Whether to include contributors in the XML. Defaults to True.

Returns:

The root XML element representing the object.

Return type:

xml.etree.ElementTree.Element

unselect_empty_groups()

Unselects empty groups from each profile in the profiles list.

This method iterates through each profile in the profiles list and calls the unselect_empty_groups method on each profile, passing the current instance as an argument.

Returns:

None

class ssg.build_yaml.BuildLoader(profiles_dir, env_yaml, product_cpes, sce_metadata_path=None)

Bases: DirectoryLoader

BuildLoader is a class that extends DirectoryLoader to handle the loading and processing of build-related YAML files, including profiles, environment configurations, product CPEs, and SCE metadata.

sce_metadata

Metadata for SCE, loaded from a JSON file.

Type:

dict

components_dir

Absolute path to the components directory.

Type:

str

rule_to_components

Mapping of rules to their respective components.

Type:

dict

export_group_to_file(filename)

Exports the loaded group to a specified file.

Parameters:

filename (str) – The name of the file to which the group will be exported.

Returns:

True if the export was successful, False otherwise.

Return type:

bool

load_components()

Loads the components from the specified components directory and maps rules to components.

This method checks if the “components_root” key is present in the environment YAML. If not, it returns None. If the key is present, it constructs the absolute path to the components directory using the “product_dir” and “components_root” values from the environment YAML. It then loads the components from this directory and creates a mapping of rules to components.

Returns:

None

class ssg.build_yaml.DirectoryLoader(profiles_dir, env_yaml, product_cpes)

Bases: object

A class to load and process security content from a directory structure.

profiles_dir

Directory containing profiles.

Type:

str

env_yaml

Environment YAML configuration.

Type:

dict

product_cpes

Product CPEs.

Type:

object

benchmark_file

Path to the benchmark file.

Type:

str

group_file

Path to the group file.

Type:

str

loaded_group

Loaded group object.

Type:

object

rule_files

List of rule file paths.

Type:

list

value_files

List of value file paths.

Type:

list

subdirectories

List of subdirectory paths.

Type:

list

all_values

Dictionary of all loaded values.

Type:

dict

all_rules

Dictionary of all loaded rules.

Type:

dict

all_groups

Dictionary of all loaded groups.

Type:

dict

parent_group

Parent group object.

Type:

object

load_benchmark_or_group(guide_directory)

Loads a given benchmark or group from the specified benchmark_file or group_file, in the context of guide_directory, profiles_dir and env_yaml.

Parameters:

guide_directory (str) – The directory containing the guide files.

Returns:

The loaded group or benchmark.

Return type:

Group or Benchmark

Raises:

ValueError – If both a .benchmark file and a .group file are found in the same directory.

process_directory_tree(start_dir, extra_group_dirs=None)

Processes the directory tree starting from the given directory.

This method collects items to load from the start directory, optionally adds extra group directories to the list of subdirectories, and then processes and recurses through the directory tree.

Parameters:

• start_dir (str) – The starting directory to process.

• extra_group_dirs (list, optional) – A list of additional directories to include in the processing. Defaults to None.

Returns:

None

process_directory_trees(directories)

Processes a list of directory trees.

Parameters:

directories (list) – A list of directory paths. The first directory in the list is considered the start directory, and the remaining directories are considered extra group directories.

Returns:

The result of processing the start directory and extra group directories.

save_all_entities(base_dir)

Saves all entities (rules, groups, values, platforms, and cpe_items) to the specified base directory.

This method creates subdirectories within the base directory for each type of entity and saves the entities if they exist. The subdirectories created are:

• rules

• groups

• values

• platforms

• cpe_items

The entities are saved using the save_entities method.

Parameters:

base_dir (str) – The base directory where the entities will be saved.

Returns:

None

save_entities(entities, destdir)

Save a list of entities to YAML files in the specified directory.

Parameters:

• entities (list) – A list of entities to be saved. Each entity must have an ‘id_’ attribute and a ‘dump_yaml’ method.

• destdir (str) – The destination directory where the YAML files will be saved.

Returns:

None

class ssg.build_yaml.Group(id_)

Bases: XCCDFEntity

Represents an XCCDF Group entity, which is a collection of rules, values, and sub-groups that can be processed and converted into various formats such as YAML and XML.

GENERIC_FILENAME

The default filename for the group.

Type:

str

KEYS

A dictionary of keys and their default values for the group.

Type:

dict

MANDATORY_KEYS

A set of keys that are mandatory for the group.

Type:

set

GENERIC_FILENAME = 'group.yml'

KEYS: Dict[str, Callable[[], Any | None]] = {'conflicts': <function Group.<lambda>>, 'cpe_platform_names': <function Group.<lambda>>, 'definition_location': <function XCCDFEntity.<lambda>>, 'description': <function Group.<lambda>>, 'groups': <function Group.<lambda>>, 'id_': <function XCCDFEntity.<lambda>>, 'inherited_platforms': <function Group.<lambda>>, 'platform': <function Group.<lambda>>, 'platforms': <function Group.<lambda>>, 'requires': <function Group.<lambda>>, 'rules': <function Group.<lambda>>, 'title': <function XCCDFEntity.<lambda>>, 'values': <function Group.<lambda>>, 'warnings': <function Group.<lambda>>}

MANDATORY_KEYS: Set[str] = {'description', 'front_matter', 'rear_matter', 'status', 'title'}

add_group(group, env_yaml=None, product_cpes=None)

Adds a group to the current object.

Parameters:

• group (str) – The group to be added.

• env_yaml (dict, optional) – Environment YAML data. Defaults to None.

• product_cpes (list, optional) – List of product CPEs. Defaults to None.

Returns:

None

add_rule(rule, env_yaml=None, product_cpes=None)

Adds a rule to the current object and processes its inherited platforms.

Parameters:

• rule (Rule) – The rule object to be added.

• env_yaml (dict, optional) – Environment-specific YAML data. Defaults to None.

• product_cpes (dict, optional) – Product-specific CPEs. Defaults to None.

Returns:

None

add_value(value)

Adds a value to the values dictionary if the value is not None.

Parameters:

value – An object with an ‘id_’ attribute that will be used as the key in the values dictionary.

Returns:

None

contains_rules(rule_ids)

Check if the specified rule IDs are present in the rules.

Parameters:

rule_ids (list) – A list of rule IDs to check for presence.

Returns:

True if all specified rule IDs are present, False otherwise.

Return type:

bool

contains_variables(variable_ids)

Check if the given variable IDs are present in the values.

Parameters:

variable_ids (list) – A list of variable IDs to check for presence.

Returns:

True if all the given variable IDs are present in the values, False otherwise.

Return type:

bool

get_not_included_components(rule_ids_list, variables_ids_list)

Identify and return the sets of rules, variables, and groups that are not included in the provided lists of rule IDs and variable IDs.

Parameters:

• rule_ids_list (list) – A list of rule IDs to check against.

• variables_ids_list (list) – A list of variable IDs to check against.

Returns:

A tuple containing three sets: - rules (set): A set of rule IDs that are not in the provided rule_ids_list. - groups (set): A set of group IDs that are not included based on the provided lists. - variables (set): A set of variable IDs that are not in the provided variables_ids_list.

Return type:

tuple

get_used_cpe_platforms(rule_ids_list)

Get the set of CPE platforms used by the given list of rule IDs.

This method collects CPE platform names from the specified rules and their inherited rules. It also includes CPE platforms from sub-groups of the given rules.

Parameters:

rule_ids_list (list) – A list of rule IDs to retrieve CPE platforms for.

Returns:

A set of CPE platform names used by the specified rules.

Return type:

set

load_entities(rules_by_id, values_by_id, groups_by_id)

Load entities into the current object’s rules, values, and groups attributes.

This method updates the current object’s rules, values, and groups attributes with the corresponding entities from the provided dictionaries (rules_by_id, values_by_id, groups_by_id). If an entity is not present in the current object’s attributes, it will be added from the provided dictionaries.

Parameters:

• rules_by_id (dict) – A dictionary containing rule entities indexed by their IDs.

• values_by_id (dict) – A dictionary containing value entities indexed by their IDs.

• groups_by_id (dict) – A dictionary containing group entities indexed by their IDs.

Returns:

None

Raises:

KeyError – If a group ID in the current object’s groups attribute is not found in the groups_by_id dictionary, the group will be removed from the current object’s groups attribute.

classmethod process_input_dict(input_contents, env_yaml, product_cpes=None)

Processes the input dictionary and updates it with additional data.

Parameters:

• cls (type) – The class type that calls this method.

• input_contents (dict) – The input dictionary containing initial data.

• env_yaml (dict) – The environment YAML data.

• product_cpes (dict, optional) – The product CPEs, defaults to None.

Returns:

The processed data dictionary with updated rules, groups, values, and platforms.

Return type:

dict

remove_rules_with_ids_not_listed(rule_ids_list)

Remove rules from the current object whose IDs are not listed in the provided rule_ids_list.

This method updates the rules attribute by filtering out rules whose IDs are not in the provided list. It also recursively applies the same filtering to all groups within the current object.

Parameters:

rule_ids_list (list) – A list of rule IDs to retain.

Returns:

None

represent_as_dict()

Represents the object as a dictionary.

This method overrides the represent_as_dict method from the superclass. It adds the rules, groups, and values attributes to the dictionary representation if they are present.

Returns:

A dictionary representation of the object with sorted keys for

rules, groups, and values if they are not empty.

Return type:

dict

to_xml_element(env_yaml=None, components_to_not_include=None)

Converts the current object to an XML element.

Parameters:

• env_yaml (dict, optional) – Environment YAML data. Defaults to None.

• components_to_not_include (dict, optional) – Components to exclude from the XML. Defaults to None. The dictionary can contain the following keys: - “rules”: A set of rule IDs to exclude. - “groups”: A set of group IDs to exclude. - “variables”: A set of variable IDs to exclude.

Returns:

The XML element representing the current object, or None if the object is in the groups_to_not_include set.

Return type:

xml.etree.ElementTree.Element

class ssg.build_yaml.LinearLoader(env_yaml, resolved_path)

Bases: object

LinearLoader is responsible for loading and managing various security content entities such as rules, profiles, groups, values, platforms, fixes, and CPE items from a specified directory structure. It also provides methods to export this content to XML format.

resolved_rules_dir

Directory path for resolved rules.

Type:

str

rules

Dictionary to store loaded rules.

Type:

dict

resolved_profiles_dir

Directory path for resolved profiles.

Type:

str

profiles

Dictionary to store loaded profiles.

Type:

dict

resolved_groups_dir

Directory path for resolved groups.

Type:

str

groups

Dictionary to store loaded groups.

Type:

dict

resolved_values_dir

Directory path for resolved values.

Type:

str

values

Dictionary to store loaded values.

Type:

dict

resolved_platforms_dir

Directory path for resolved platforms.

Type:

str

platforms

Dictionary to store loaded platforms.

Type:

dict

fixes_dir

Directory path for fixes.

Type:

str

fixes

Dictionary to store loaded fixes.

Type:

dict

resolved_cpe_items_dir

Directory path for resolved CPE items.

Type:

str

cpe_items

Dictionary to store loaded CPE items.

Type:

dict

benchmark

Loaded benchmark object.

Type:

Benchmark

env_yaml

Environment YAML configuration.

Type:

dict

product_cpes

Product CPEs object.

Type:

ProductCPEs

off_ocil

Flag to indicate if OCIL should be turned off.

Type:

bool

add_fixes_to_rules()

Adds fixes to the rules in the rules dictionary.

Iterates over the items in the fixes dictionary and adds the corresponding fixes to the rules in the rules dictionary.

Returns:

None

Raises:

KeyError – If a rule_id in fixes does not exist in rules.

export_benchmark_to_file(filename)

Exports the benchmark data to a specified file.

This method registers the necessary namespaces and then writes the benchmark data to the given filename using the environment YAML configuration.

Parameters:

filename (str) – The path to the file where the benchmark data will be exported.

Returns:

True if the export was successful, False otherwise.

Return type:

bool

export_benchmark_to_xml(rule_and_variables_dict, ignore_single_rule_profiles)

Exports the benchmark data to an XML format.

Parameters:

• rule_and_variables_dict (dict) – A dictionary containing rules and variables.

• ignore_single_rule_profiles (bool) – All profiles that contain “single_rule_profile: true” will be skipping.

Returns:

The benchmark data in XML format.

Return type:

str

export_ocil_to_file(filename)

Exports the OCIL data to an XML file.

This method converts the OCIL data to an XML format and writes it to the specified file. If the conversion to XML results in a None root, the method returns without writing to the file.

Parameters:

filename (str) – The path to the file where the XML data will be written.

Returns:

None

export_ocil_to_xml(benchmark=None)

Exports OCIL content to an XML format.

Parameters:

benchmark (optional) – The benchmark object containing rules to be exported. If not provided, the instance’s benchmark attribute is used.

Returns:

The root element of the generated OCIL XML tree.

Returns None if OCIL is turned off or no rules are found.

Return type:

xml.etree.ElementTree.Element

find_first_groups_ids(start_dir)

Finds the IDs of the first-level groups in the specified directory.

This method searches for all “group.yml” files located in the immediate subdirectories of the given start directory and extracts the names of these subdirectories as group IDs.

Parameters:

start_dir (str) – The directory to start searching from.

Returns:

A list of group IDs (names of subdirectories containing “group.yml”).

Return type:

list

get_benchmark_xml()

Converts the benchmark data to an XML element.

Returns:

The XML representation of the benchmark data.

Return type:

xml.etree.ElementTree.Element

get_benchmark_xml_by_profile(rule_and_variables_dict)

Generates benchmark XML for each profile in the benchmark.

This method iterates over all profiles in the benchmark and generates the corresponding benchmark XML for each profile using the provided rule and variables dictionary.

Parameters:

rule_and_variables_dict (dict) – A dictionary containing rules and variables to be used for generating the benchmark XML.

Yields:

tuple – A tuple containing the profile ID (str) and the corresponding benchmark XML (str).

Raises:

Exception – If the benchmark is not loaded before calling this method.

load_benchmark(directory)

Loads the benchmark data from the specified directory.

This method performs the following steps: 1. Loads the benchmark from a YAML file located in the given directory. 2. Adds profiles from the resolved profiles directory. 3. Adds groups to the benchmark based on the first group IDs found in the directory. 4. Drops rules that are not included in any profile. 5. Unselects empty groups from the benchmark.

Parameters:

directory (str) – The directory from which to load the benchmark data.

Returns:

None

Raises:

KeyError – If a group ID is not found in the compiled and loaded groups.

load_compiled_content()

Loads and compiles various content entities from specified directories into the appropriate attributes.

This method performs the following steps: 1. Loads CPEs from the directory tree specified by self.resolved_cpe_items_dir and

updates self.product_cpes.

2. Loads compiled remediations from the directory specified by self.fixes_dir and assigns them to self.fixes.

3. Loads rule entities from YAML files in the directory specified by self.resolved_rules_dir and updates self.rules.

4. Loads group entities from YAML files in the directory specified by self.resolved_groups_dir and updates self.groups.

5. Loads value entities from YAML files in the directory specified by self.resolved_values_dir and updates self.values.

6. Loads platform entities from YAML files in the directory specified by self.resolved_platforms_dir and updates self.platforms.

7. Assigns the loaded platforms to self.product_cpes.platforms.

8. For each group in self.groups, loads associated rules, values, and sub-groups.

Returns:

None

Raises:

• FileNotFoundError – If any of the specified directories or files do not exist.

• ValueError – If there is an issue with the content of the YAML files.

load_entities_by_id(filenames, destination, cls)

Loads entities from a list of JSON files and stores them in a destination dictionary by their ID.

Parameters:

• filenames (list of str) – List of file paths to JSON files.

• destination (dict) – Dictionary to store the loaded entities, keyed by their ID.

• cls (type) – Class type that has a from_yaml method to create an instance from a JSON file.

Returns:

None

class ssg.build_yaml.Platform(id_)

Bases: XCCDFEntity

Represents a platform entity in the XCCDF standard.

KEYS

Dictionary of keys with default lambda functions.

Type:

dict

MANDATORY_KEYS

List of mandatory keys for the platform.

Type:

list

prefix

Prefix for the platform.

Type:

str

ns

Namespace for the platform.

Type:

str

KEYS: Dict[str, Callable[[], Any | None]] = {'ansible_conditional': <function Platform.<lambda>>, 'bash_conditional': <function Platform.<lambda>>, 'definition_location': <function XCCDFEntity.<lambda>>, 'id_': <function XCCDFEntity.<lambda>>, 'name': <function Platform.<lambda>>, 'original_expression': <function Platform.<lambda>>, 'title': <function XCCDFEntity.<lambda>>, 'xml_content': <function Platform.<lambda>>}

MANDATORY_KEYS: Set[str] = {'ansible_conditional', 'bash_conditional', 'name', 'original_expression', 'xml_content'}

classmethod from_compiled_json(json_file_path, env_yaml=None, product_cpes=None)

classmethod from_text(expression, product_cpes)

Creates a platform object from a given text expression and product CPEs.

Parameters:

• cls – The class itself.

• expression (str) – The text expression to parse.

• product_cpes (ProductCPEs) – The product CPEs to use for parsing and resolving CPE items.

Returns:

A platform object with the parsed and enriched CPE information,

or None if product_cpes is empty.

Return type:

platform (Platform)

classmethod from_yaml(yaml_file, env_yaml=None, product_cpes=None)

Creates a Platform object from a YAML file.

Parameters:

• yaml_file (str) – Path to the YAML file.

• env_yaml (dict, optional) – Environment YAML data. Defaults to None.

• product_cpes (ProductCPEs, optional) – Product CPEs object for restoring the original test object. Defaults to None.

Returns:

A Platform object created from the YAML file.

Return type:

Platform

get_fact_refs()

Retrieve fact references from the test symbols.

Returns:

A list of symbols from the test object.

Return type:

list

get_remediation_conditional(language)

Returns the remediation conditional based on the specified language.

Parameters:

language (str) – The remediation language. Supported values are “bash” and “ansible”.

Returns:

The corresponding remediation conditional for the specified language.

Return type:

str

Raises:

AttributeError – If an invalid remediation language is specified.

get_xml()

Generates an XML string representation of the platform.

This method creates an XML element for the platform with the appropriate namespace and sets its ‘id’ attribute to the platform’s name. If the platform contains only a single CPE name, it creates a logical test element with an ‘AND’ operator and ‘false’ negate attribute to adhere to the CPE specification. The logical test element or the test element is then appended to the platform element.

Returns:

A string representation of the XML for the platform.

Return type:

str

ns = 'http://cpe.mitre.org/language/2.0'

prefix = 'cpe-lang'

to_xml_element()

Converts the stored XML content into an XML element.

Returns:

The root element of the parsed XML content.

Return type:

xml.etree.ElementTree.Element

update_conditional_from_cpe_items(language, product_cpes)

Updates the conditional statements for the specified language based on CPE items.

This method enriches the test object with CPE information and then generates the appropriate conditional statements for the specified language.

Parameters:

• language (str) – The language for which to generate the conditional statements. Supported values are “bash” and “ansible”.

• product_cpes (list) – A list of CPE items to enrich the test object with.

Returns:

None

Raises:

RuntimeError – If the specified language is not supported.

class ssg.build_yaml.Rule(id_)

Bases: XCCDFEntity, Templatable

Represents an XCCDF Rule entity with various attributes and methods for handling rule data, validation, and conversion.

KEYS

Default values for various rule attributes.

Type:

dict

MANDATORY_KEYS

Set of mandatory keys for a rule.

Type:

set

GENERIC_FILENAME

Default filename for a rule.

Type:

str

ID_LABEL

Label for rule ID.

Type:

str

PRODUCT_REFERENCES

Tuple of product-specific reference types.

Type:

tuple

GENERIC_FILENAME = 'rule.yml'

ID_LABEL = 'rule_id'

KEYS: Dict[str, Callable[[], Any | None]] = {'bash_conditional': <function Rule.<lambda>>, 'check_type': <function Rule.<lambda>>, 'checktext': <function Rule.<lambda>>, 'components': <function Rule.<lambda>>, 'conflicts': <function Rule.<lambda>>, 'control_references': <function Rule.<lambda>>, 'cpe_platform_names': <function Rule.<lambda>>, 'definition_location': <function XCCDFEntity.<lambda>>, 'description': <function Rule.<lambda>>, 'expression': <function Rule.<lambda>>, 'failure_reason': <function Rule.<lambda>>, 'fixes': <function Rule.<lambda>>, 'fixtext': <function Rule.<lambda>>, 'id_': <function XCCDFEntity.<lambda>>, 'identifiers': <function Rule.<lambda>>, 'inherited_cpe_platform_names': <function Rule.<lambda>>, 'inherited_platforms': <function Rule.<lambda>>, 'inputs': <function Rule.<lambda>>, 'ocil': <function Rule.<lambda>>, 'ocil_clause': <function Rule.<lambda>>, 'oval_external_content': <function Rule.<lambda>>, 'platform': <function Rule.<lambda>>, 'platforms': <function Rule.<lambda>>, 'policy_specific_content': <function Rule.<lambda>>, 'rationale': <function Rule.<lambda>>, 'references': <function Rule.<lambda>>, 'requires': <function Rule.<lambda>>, 'sce_metadata': <function Rule.<lambda>>, 'severity': <function Rule.<lambda>>, 'srg_requirement': <function Rule.<lambda>>, 'template': <function Templatable.<lambda>>, 'title': <function XCCDFEntity.<lambda>>, 'vuldiscussion': <function Rule.<lambda>>, 'warnings': <function Rule.<lambda>>}

MANDATORY_KEYS: Set[str] = {'description', 'rationale', 'severity', 'title'}

PRODUCT_REFERENCES = ('stigid', 'cis')

add_control_reference(ref_type, ref_value)

Adds a control reference to the rule.

Parameters:

• ref_type (str) – The type of the control reference.

• ref_value (str) – The value of the control reference.

Returns:

None

Raises:

ValueError – If the rule already contains the specified reference type and value.

add_fixes(fixes)

Adds a list of fixes to the current instance.

Parameters:

fixes (list) – A list of fixes to be added.

Returns:

None

add_stig_references(stig_references)

Adds STIG references to the object’s references.

This method looks up STIG references based on the STIG IDs present in the object’s references. If any matching references are found, they are added to the object’s references under the key “stigref”.

Parameters:

stig_references (dict) – A dictionary mapping STIG IDs to their corresponding references.

Returns:

None

find_policy_specific_content(rule_root)

Finds and returns a set of policy-specific YAML files within a given rule directory.

Parameters:

rule_root (str) – The root directory of the rule to search for policy-specific content.

Returns:

A set of file paths to the policy-specific YAML files found within the rule

directory.

Return type:

set

classmethod from_yaml(yaml_file, env_yaml=None, product_cpes=None, sce_metadata=None)

Creates a Rule object from a YAML file.

Parameters:

• yaml_file (str) – Path to the YAML file containing the rule definition.

• env_yaml (dict, optional) – Environment-specific YAML data. Defaults to None.

• product_cpes (list, optional) – List of product CPEs. Defaults to None.

• sce_metadata (dict, optional) – SCE metadata dictionary. Defaults to None.

Returns:

An instance of the Rule class populated with data from the YAML file.

Return type:

Rule

Notes

• Converts platforms from list to set.

• Splits references from comma-separated strings to lists.

• Ensures rule.platform is included in rule.platforms.

• Loads policy-specific content if not already defined.

• Adds SCE metadata if available.

• Validates identifiers and references.

get_template_context(env_yaml)

Generates the template context for the given environment YAML.

This method overrides the parent class’s get_template_context method to include additional context specific to this class.

Parameters:

env_yaml (dict) – The environment YAML data.

Returns:

The template context with additional identifiers if available.

Return type:

dict

load_policy_specific_content(rule_filename, env_yaml)

Loads policy-specific content for a given rule.

This method finds and reads policy-specific content files related to the provided rule filename and environment YAML. The content is then stored in the instance variable policy_specific_content.

Parameters:

• rule_filename (str) – The filename of the rule for which to load policy-specific content.

• env_yaml (dict) – The environment YAML data.

Returns:

None

make_refs_and_identifiers_product_specific(product)

Adjusts the rule’s references and identifiers to be product-specific.

This method modifies the rule’s references and identifiers by appending a product-specific suffix to them. It ensures that general references do not contain product-specific identifiers and raises an error if such identifiers are found.

Parameters:

product (str) – The product identifier to be appended to references and identifiers.

Returns:

None

Raises:

ValueError – If an unexpected reference identifier without a product qualifier is found, or if there is an error processing the references or identifiers.

merge_control_references()

Merges control references into the main references dictionary.

This method iterates over the control references and adds them to the main references dictionary. If a reference type already exists in the main references, the control references of that type are appended to the existing list. If the reference type does not exist, it is added to the main references dictionary.

Returns:

None

normalize(product)

Normalize the given product by making references and identifiers product-specific and applying product-specific templates.

Parameters:

product (str) – The product to be normalized.

Returns:

None

Raises:

RuntimeError – If an error occurs during normalization, an exception is raised with a message indicating the rule and the error message.

read_policy_specific_content(env_yaml, files)

Reads policy-specific content from a list of files and returns a dictionary of keys.

Parameters:

• env_yaml (dict) – A dictionary containing environment-specific YAML data.

• files (list) – A list of file paths to read the policy-specific content from.

Returns:

A dictionary where the keys are policy identifiers and the values are the

corresponding YAML data.

Return type:

dict

read_policy_specific_content_file(env_yaml, filename)

Reads and processes a policy-specific content file.

Parameters:

• env_yaml (dict) – The environment variables and macros for YAML expansion.

• filename (str) – The path to the policy-specific content file.

Returns:

The processed YAML data from the content file.

Return type:

dict

to_ocil()

Converts the rule to OCIL format.

This method generates an OCIL questionnaire, boolean question test action, and boolean question for the rule. It ensures that the rule has the necessary OCIL and OCIL clause information, and processes the OCIL content to remove HTML and XML tags where necessary.

Returns:

A tuple containing the OCIL questionnaire, boolean question test action, and

boolean question elements.

Return type:

tuple

Raises:

ValueError – If the rule does not have OCIL or OCIL clause information.

to_xml_element(env_yaml=None)

Converts the rule object to an XML element.

Parameters:

env_yaml (dict, optional) – A dictionary containing environment-specific YAML data. If provided, it will be used to fetch reference URIs. Defaults to None.

Returns:

An XML element representing the rule.

Return type:

xml.etree.ElementTree.Element

triage_policy_specific_content(product_name, filenames)

Organizes filenames by policy based on the product name and specific criteria.

Parameters:

• product_name (str) – The name of the product.

• filenames (list) – A list of filenames to be triaged.

Returns:

A dictionary where the keys are policy names and the values are filenames that

match the criteria for storage.

Return type:

dict

validate_identifiers(yaml_file)

Validates the identifiers in the given YAML file.

Parameters:

yaml_file (str) – The path to the YAML file being validated.

Returns:

None

Raises:

ValueError – If the identifiers section is empty, if any identifier or value is not a string, if any identifier value is empty, if a CCE identifier format is invalid, or if a CCE identifier value is not a valid checksum.

validate_references(yaml_file)

Validates the references section of a YAML file.

Parameters:

yaml_file (str) – The path to the YAML file being validated.

Returns:

None

Raises:

• ValueError – If the references section is empty.

• ValueError – If any reference type is not a string.

• ValueError – If any reference value is empty.

• ValueError – If any reference contains leading or trailing whitespace.

class ssg.build_yaml.Value(id_)

Bases: XCCDFEntity

Represents an XCCDF Value entity.

KEYS

A dictionary of default values for various attributes.

Type:

dict

MANDATORY_KEYS

A set of keys that are mandatory for the Value entity.

Type:

set

KEYS: Dict[str, Callable[[], Any | None]] = {'definition_location': <function XCCDFEntity.<lambda>>, 'description': <function Value.<lambda>>, 'id_': <function XCCDFEntity.<lambda>>, 'interactive': <function Value.<lambda>>, 'operator': <function Value.<lambda>>, 'options': <function Value.<lambda>>, 'title': <function XCCDFEntity.<lambda>>, 'type': <function Value.<lambda>>, 'warnings': <function Value.<lambda>>}

MANDATORY_KEYS: Set[str] = {'description', 'title', 'type'}

classmethod from_yaml(yaml_file, env_yaml=None, product_cpes=None)

Create an instance of the class from a YAML file.

Parameters:

• yaml_file (str) – Path to the YAML file.

• env_yaml (str, optional) – Path to an environment YAML file. Defaults to None.

• product_cpes (str, optional) – Product CPEs information. Defaults to None.

Returns:

An instance of the class created from the YAML file.

Return type:

Value

classmethod process_input_dict(input_contents, env_yaml, product_cpes=None)

Processes the input dictionary for a given environment YAML and optional product CPEs.

Parameters:

• input_contents (dict) – The input dictionary containing various parameters.

• env_yaml (dict) – The environment YAML configuration.

• product_cpes (optional) – Product CPEs, if any.

Returns:

Processed data dictionary with validated and possibly modified contents.

Return type:

dict

Raises:

ValueError – If the operator in the input data is not one of the expected possible operators.

to_xml_element()

Converts the current object to an XML element.

This method creates an XML element representing the current object using the XCCDF 1.2 namespace. It sets attributes and child elements based on the object’s properties.

Returns:

The XML element representing the object.

Return type:

xml.etree.ElementTree.Element

ssg.build_yaml.add_benchmark_metadata(element, include_contributors)

Adds benchmark metadata to an XML element.

This function appends metadata information to the provided XML element, including publisher, creator, contributors, and source information.

Parameters:

• element (xml.etree.ElementTree.Element) – The XML element to which the metadata will be added.

• include_contributors (bool) – A flag indicating whether to include contributors in the metadata.

Returns:

None

ssg.build_yaml.add_nondata_subelements(element, subelement, attribute, attr_data)

Add multiple iterations of a subelement that contains an attribute but no data.

This function creates subelements under a given XML element. Each subelement will have a specified attribute set to a value from the provided list of attribute data.

Parameters:

• element (xml.etree.ElementTree.Element) – The parent XML element to which subelements will be added.

• subelement (str) – The tag name of the subelements to be created.

• attribute (str) – The name of the attribute to be set on each subelement.

• attr_data (list) – A list of values to be assigned to the specified attribute of each subelement.

ssg.build_yaml.add_platform_if_not_defined(platform, product_cpes)

Adds a platform to the product_cpes if it is not already defined.

Parameters:

• platform (Platform) – The platform to be added.

• product_cpes (ProductCPEs) – The product CPEs object containing platforms.

Returns:

The existing platform if it was already defined, otherwise the newly added platform.

Return type:

Platform

ssg.build_yaml.add_reference_elements(element, references, ref_uri_dict)

Adds reference elements to an XML element based on provided references and their corresponding URIs.

Parameters:

• element (xml.etree.ElementTree.Element) – The XML element to which reference elements will be added.

• references (dict) – A dictionary where keys are reference types (e.g., ‘srg’) and values are lists of reference values.

• ref_uri_dict (dict) – A dictionary mapping reference types to their corresponding URIs.

Returns:

None

Raises:

ValueError – If an SRG reference does not have a defined URI or if an unknown reference type is encountered.

ssg.build_yaml.add_reference_title_elements(benchmark_el, env_yaml)

Adds reference title elements to the given benchmark element.

This function creates and appends reference elements to the provided benchmark element. The references are sourced from the env_yaml if provided, otherwise from the default SSG_REF_URIS.

Parameters:

• benchmark_el (xml.etree.ElementTree.Element) – The benchmark element to which reference elements will be added.

• env_yaml (dict) – A dictionary containing reference URIs. If None, the default SSG_REF_URIS will be used.

Returns:

None

ssg.build_yaml.add_warning_elements(element, warnings)

Adds warning elements to the given XML element.

This function iterates over a list of warning dictionaries and adds each warning as a sub-element to the provided XML element. Each dictionary in the warnings list should contain a single key-value pair, where the key represents the warning category and the value represents the warning text.

Parameters:

• element (xml.etree.ElementTree.Element) – The XML element to which the warning elements will be added.

• warnings (list of dict) – A list of dictionaries, each containing a single key-value pair representing the warning category and text.

Returns:

None

ssg.build_yaml.check_warnings(xccdf_structure)

Checks the warnings in the given xccdf_structure.

This function iterates through the warnings in the xccdf_structure and ensures that each warning dictionary contains exactly one key/value pair. If a warning dictionary contains more than one key/value pair, a ValueError is raised with an appropriate message.

Parameters:

xccdf_structure (object) – An object that contains a list of warning dictionaries under the attribute ‘warnings’.

Returns:

None

Raises:

ValueError – If any warning dictionary contains more than one key/value pair.

ssg.build_yaml.noop_rule_filterfunc(rule)

ssg.build_yaml.reorder_according_to_ordering(unordered, ordering, regex=None)

Reorders a list of items according to a specified ordering.

Parameters:

• unordered (list) – The list of unordered items.

• ordering (list) – The list of items specifying the desired order.

• regex (str, optional) – A regex pattern to filter items to be ordered. If None, a pattern is created from the ordering list.

Returns:

A list of items ordered according to the ordering list, followed by any remaining

items sorted alphabetically.

Return type:

list

ssg.build_yaml.restore_original_test_object(platform, product_cpes)

ssg.build_yaml.rule_filter_from_def(filterdef)

Creates a filter function based on the provided filter definition.

Parameters:

filterdef (str or None) – A string containing a Python expression that will be used to filter rules. If None or an empty string is provided, a no-operation filter function is returned.

Returns:

A function that takes a rule object and evaluates the filter definition against

the rule’s attributes. If the filter definition is None or an empty string, a no-operation filter function is returned.

Return type:

function

Note

The filter function uses eval to evaluate the filter definition. For security reasons, only the rule’s attributes are exposed to the evaluation context, and Python built-ins are not available.

ssg.cce module

Common functions for processing CCE (Common Configuration Enumeration) in SSG

class ssg.cce.CCEFile(project_root=None)

Bases: object

A class to represent a CCE (Common Configuration Enumeration) file.

project_root

The root directory of the project.

Type:

str

property absolute_path

line_isnt_cce(cce, line)

line_to_cce(line)

random_cce()

Selects a random CCE (Common Configuration Enumeration) from the list of CCEs.

Reads the list of CCEs, shuffles them randomly, and returns the first CCE from the shuffled list after stripping any leading or trailing whitespace.

Returns:

A randomly selected and stripped CCE.

Return type:

str

read_cces()

Reads the CCEs (Common Configuration Enumeration) from a file and validates them.

This method opens the file specified by self.absolute_path, reads its contents line by line, and checks each line to ensure it is a valid CCE value. If an invalid CCE is detected, a RuntimeError is raised with an appropriate error message.

Returns:

A list of valid CCE values read from the file.

Return type:

list

Raises:

RuntimeError – If an invalid CCE value is detected in the file.

remove_cce_from_file(cce)

Removes lines containing the specified CCE (Common Configuration Enumeration) from a file.

Parameters:

cce (str) – The CCE identifier to be removed from the file.

The method reads the file, filters out lines containing the specified CCE, and writes the remaining lines back to the file.

class ssg.cce.RedhatCCEFile(project_root=None)

Bases: CCEFile

RedhatCCEFile is a subclass of CCEFile that represents a file containing Red Hat Common Configuration Enumeration (CCE) data.

Properties:

absolute_path (str): The absolute path to the Red Hat CCE file, which is located in the

“shared/references” directory within the project root and named “cce-redhat-avail.txt”.

property absolute_path

class ssg.cce.SLE12CCEFile(project_root=None)

Bases: CCEFile

SLE12CCEFile is a subclass of CCEFile that represents a file containing SLE12 Common Configuration Enumeration (CCE) data.

This class provides a property to get the absolute path of the SLE12 CCE file.

Properties:

absolute_path (str): The absolute path to the SLE12 CCE file, which is located in the

“shared/references” directory.

property absolute_path

class ssg.cce.SLE15CCEFile(project_root=None)

Bases: CCEFile

SLE15CCEFile is a subclass of CCEFile that represents a file containing SLE15 Common Configuration Enumeration (CCE) data.

This class provides a property to get the absolute path of the SLE12 CCE file.

Properties:

absolute_path (str): The absolute path to the SLE15 CCE file, which is located in the

“shared/references” directory.

property absolute_path

class ssg.cce.SLE16CCEFile(project_root=None)

Bases: CCEFile

SLE16CCEFile is a subclass of CCEFile that represents a file containing SLE16 Common Configuration Enumeration (CCE) data.

This class provides a property to get the absolute path of the SLE12 CCE file.

Properties:

absolute_path (str): The absolute path to the SLE15 CCE file, which is located in the

“shared/references” directory.

property absolute_path

ssg.cce.is_cce_format_valid(cceid)

Check if the given CCE ID is in a valid format.

A valid CCE ID must be in one of the following formats: - ‘CCE-XXXX-X’ - ‘CCE-XXXXX-X’

where each ‘X’ is a digit, and the final ‘X’ is a check-digit.

Parameters:

cceid (str) – The CCE ID to validate.

Returns:

True if the CCE ID is in a valid format, False otherwise.

Return type:

bool

ssg.cce.is_cce_value_valid(cceid)

Validates a CCE (Common Configuration Enumeration) identifier using Luhn’s algorithm.

This function removes non-digit characters from the CCE identifier and then applies Luhn’s algorithm to determine if the identifier is valid.

Parameters:

cceid (str) – The CCE identifier to validate.

Returns:

True if the CCE identifier is valid, False otherwise.

Return type:

bool

References

For context, see: https://github.com/ComplianceAsCode/content/issues/3044#issuecomment-420844095

ssg.checks module

Common functions for processing Checks in SSG

ssg.checks.get_content_ref_if_exists_and_not_remote(check)

Given an OVAL check element, examine the xccdf_ns:check-content-ref.

If the check-content-ref element exists and it isn’t remote, return it.

Parameters:

check (Element) – An OVAL check element to be examined.

Returns:

The check-content-ref element if it exists and is not remote, otherwise None.

Return type:

Element or None

See also

is_content_href_remote: Function to determine if the content reference is remote.

ssg.checks.get_oval_contents(rule_obj, oval_id)

Returns the tuple (path, contents) of the check described by the given oval_id or product.

Parameters:

• rule_obj (object) – The rule object containing the OVAL definitions.

• oval_id (str) – The identifier of the OVAL check.

Returns:

A tuple containing the path to the OVAL file and its contents.

Return type:

tuple

ssg.checks.get_oval_path(rule_obj, oval_id)

Returns the full path to the OVAL check file for the given rule object and OVAL ID.

Parameters:

• rule_obj (dict) – A dictionary containing rule information. It must include the keys ‘dir’, ‘id’, and ‘ovals’.

• oval_id (str) – A string representing the ID of the OVAL check file. If it does not end with “.xml”, the extension will be appended.

Returns:

The full path to the OVAL check file.

Return type:

str

Raises:

ValueError – If the rule_obj is malformed or if the oval_id is unknown for the given rule.

ssg.checks.is_content_href_remote(check_content_ref)

Given an OVAL check-content-ref element, examine the ‘href’ attribute.

Parameters:

check_content_ref (Element) – An XML element representing the OVAL check-content-ref.

Returns:

True if the ‘href’ attribute starts with ‘http://’ or ‘https://’, otherwise False.

Return type:

bool

Raises:

RuntimeError – If the ‘href’ attribute does not exist in the check_content_ref element.

ssg.checks.set_applicable_platforms(oval_contents, new_platforms)

Modifies the given OVAL contents to update the platforms to the new platforms.

Parameters:

• oval_contents (list of str) – The original OVAL content lines.

• new_platforms (list of str) – The new platforms to be set in the OVAL content.

Returns:

The modified OVAL content lines with updated platforms.

Return type:

list of str

ssg.components module

Common functions for processing Components in SSG

class ssg.components.Component(filepath)

Bases: object

A class to represent a component. With regards to the content, a component usually represents a piece of software.

name

The name of the component.

Type:

str

rules

A list of rules associated with the component.

Type:

list

packages

A list of packages associated with the component.

Type:

list

templates

A list of templates associated with the component. Defaults to an empty list.

Type:

list, optional

groups

A list of groups associated with the component. Defaults to an empty list.

Type:

list, optional

changelog

A list of changelog entries for the component. Defaults to an empty list.

Type:

list, optional

ssg.components.get_rule_to_components_mapping(components)

Generates a mapping from rule IDs to component names.

Parameters:

components (dict) – A dictionary where the keys are component names and the values are component objects. Each component object is expected to have a ‘rules’ attribute (a list of rule IDs) and a ‘name’ attribute (the name of the component).

Returns:

A dictionary where the keys are rule IDs and the values are lists of component names

that include the corresponding rule ID.

Return type:

dict

ssg.components.group_component_mapping(components)

Groups components by their associated groups.

Parameters:

components (dict) – A dictionary where keys are component names and values are dictionaries containing component attributes, including a “groups” key.

Returns:

A dictionary where keys are group names and values are lists of component names

that belong to each group.

Return type:

dict

ssg.components.load(components_dir)

Load components from a specified directory.

Parameters:

components_dir (str) – The directory path containing component files.

Returns:

A dictionary where the keys are component names and the values are component objects.

Return type:

dict

ssg.components.package_component_mapping(components)

Maps the given components to their respective packages.

Parameters:

components (dict) – A dictionary where keys are component names and values are component details.

Returns:

A dictionary where keys are package names and values are lists of components

associated to those packages.

Return type:

dict

ssg.components.rule_component_mapping(components)

Maps the given components to their corresponding rules.

Parameters:

components (dict) – A dictionary where keys are component names and values are component details.

Returns:

A dictionary where keys are rule names and values are the corresponding components.

Return type:

dict

ssg.components.template_component_mapping(components)

Maps the given components to their corresponding templates.

Parameters:

components (dict) – A dictionary where keys are component names and values are component details.

Returns:

A dictionary where keys are template names and values are lists of components that

use those templates.

Return type:

dict

ssg.content_diff module

class ssg.content_diff.StandardContentDiffer(old_content, new_content, rule_id, show_diffs, rule_diffs, only_rules, output_dir)

Bases: object

compare_check_file_names(system, identifier, old_check_file_name, new_check_file_name)

compare_check_ids(system, identifier, old_check_id, new_check_id)

compare_checks(old_rule, new_rule, system, identifier)

compare_ocils(old_ocil_doc, old_ocil_id, new_ocil_doc, new_ocil_id, identifier)

compare_ovals(old_oval_def_doc, old_oval_def_id, new_oval_def_doc, new_oval_def_id, identifier)

compare_platforms(old_rule, new_rule, old_benchmark, new_benchmark, identifier)

compare_remediations(old_rule, new_rule, remediation_type, identifier)

compare_rule(old_rule, new_rule, identifier)

compare_rule_texts(old_rule, new_rule, identifier)

compare_rules(old_benchmark, new_benchmark)

generate_diff_text(old_r, new_r, fromfile='old data stream', tofile='new data stream', n=3)

get_check_docs(system, identifier, old_check_file_name, new_check_file_name)

output_diff(identifier, diff, mode='a')

serialize_elements(elements)

class ssg.content_diff.StigContentDiffer(old_content, new_content, rule_id, show_diffs, rule_diffs, only_rules, output_dir)

Bases: StandardContentDiffer

check_for_new_rules(rules_in_new_benchmark, old_rule_mapping)

compare_existing_rules(new_benchmark, old_benchmark, rules_in_old_benchmark, new_rule_mapping)

compare_rules(old_benchmark, new_benchmark)

get_stig_rule_SV(sv_rule_id)

ssg.controls module

Common functions for processing Controls in SSG

class ssg.controls.Control

Bases: SelectionHandler, XCCDFEntity

Represents a control entity with various attributes and methods to handle control-related operations.

KEYS

A dictionary defining the expected keys and their types.

Type:

dict

MANDATORY_KEYS

A set of keys that are mandatory for a control.

Type:

set

KEYS: Dict[str, Callable[[], Any | None]] = {'artifact_description': <class 'str'>, 'automated': <class 'str'>, 'check': <class 'str'>, 'controls': <class 'list'>, 'description': <class 'str'>, 'fixtext': <class 'str'>, 'id': <class 'str'>, 'levels': <class 'list'>, 'mitigation': <class 'str'>, 'notes': <class 'str'>, 'original_title': <class 'str'>, 'rationale': <class 'str'>, 'related_rules': <class 'list'>, 'rules': <class 'list'>, 'status': <function Control.<lambda>>, 'status_justification': <class 'str'>, 'tickets': <class 'list'>, 'title': <class 'str'>}

MANDATORY_KEYS: Set[str] = {'title'}

add_references(reference_type, rules)

Adds references to the specified rules based on the given reference type.

Parameters:

• reference_type (str) – The type of reference to add.

• rules (dict) – A dictionary of rules where the key is the rule id and the value is the rule object.

Raises:

ValueError – If there is a duplicate listing of a rule in the control.

classmethod from_control_dict(control_dict, env_yaml=None, default_level=None)

Create a control instance from a dictionary of control attributes.

Parameters:

• cls – The class itself.

• control_dict (dict) – A dictionary containing control attributes.

• env_yaml (optional) – Environment YAML configuration (default is None).

• default_level (list, optional) – Default level for the control (default is [“default”]).

Returns:

An instance of the control class populated with attributes from the control_dict.

Raises:

ValueError – If the ‘automated’ key has an invalid value or if ‘levels’ is not a list.

represent_as_dict()

Represents the Control object as a dictionary.

This method overrides the represent_as_dict method from the superclass. It adds the rules and controls attributes to the dictionary representation.

Returns:

A dictionary representation of the Control object, including

the rules and controls attributes.

Return type:

dict

class ssg.controls.ControlsManager(controls_dirs: List[str], env_yaml=None, existing_rules=None)

Bases: object

Manages the loading, processing, and saving of control policies.

controls_dirs

The directories where control policy files are located.

Type:

List[str]

env_yaml

The environment YAML file.

Type:

str, optional

existing_rules

Existing rules to check against.

Type:

dict, optional

policies

A dictionary of loaded policies.

Type:

dict

add_references(rules)

Add references to the policies and unify them under a single attribute.

This method first adds all control references into a separate attribute for each policy. Then, it unifies them under the references attribute. This allows multiple control files to add references of the same type while still tracking what references already existed in the rule.

Parameters:

rules (dict) – A dictionary containing the rules to which references will be added.

check_all_rules_exist()

Checks if all rules exist for each policy.

This method iterates through all policies and calls the check_all_rules_exist method on each policy, passing the existing rules as an argument. If existing_rules is None, the method returns immediately.

Returns:

None

get_all_controls(policy_id)

Retrieve all controls associated with a given policy.

Parameters:

policy_id (str) – The unique identifier of the policy.

Returns:

A list of control objects associated with the specified policy.

Return type:

list

get_all_controls_dict(policy_id: str) → Dict[str, Control]

Retrieve all controls for a given policy as a dictionary.

Parameters:

policy_id (str) – The unique identifier of the policy.

Returns:

A dictionary where the keys are control IDs and the values are lists

of controls.

Return type:

Dict[str, list]

get_all_controls_of_level(policy_id: str, level_id: str) → List[Control]

Retrieve all controls associated with a specific policy and level, including inherited levels.

This method fetches all controls for a given policy and filters them based on the specified level and its ancestor levels. Controls from higher levels can override variables defined in lower levels.

Parameters:

• policy_id (str) – The unique identifier of the policy.

• level_id (str) – The unique identifier of the level within the policy.

Returns:

A list of controls that are eligible for the specified level, with variables

from higher levels overriding those from lower levels.

Return type:

list

get_control(policy_id, control_id)

Retrieve a specific control from a policy.

Parameters:

• policy_id (str) – The unique identifier of the policy.

• control_id (str) – The unique identifier of the control within the policy.

Returns:

The control object associated with the given control_id within the specified policy.

Return type:

Control

Raises:

• PolicyNotFoundError – If the policy with the given policy_id does not exist.

• ControlNotFoundError – If the control with the given control_id does not exist within the policy.

load()

Load policies from YAML files in the controls directory.

This method checks if the controls directory exists. If it does, it loads all YAML files in the directory, creates Policy objects from them, and stores them in the self.policies dictionary using their IDs as keys. After loading all policies, it checks that all rules exist and resolves controls.

Returns:

None

load_compiled()

policies: Dict

remove_selections_not_known(known_rules)

Remove selections from policies that are not in the known rules.

This method iterates over all policies and removes any selections that are not present in the provided known rules set.

Parameters:

known_rules (iterable) – An iterable of known rule identifiers.

resolve_controls()

Resolves controls for each policy in the policies dictionary.

Iterates through each policy and its associated controls, calling the _resolve_control method for each control.

Returns:

None

save_everything(output_dir)

Saves all policies to the specified output directory in JSON format.

Parameters:

output_dir (str) – The directory where the policy files will be saved.

Returns:

None

exception ssg.controls.InvalidStatus

Bases: Exception

class ssg.controls.Level

Bases: XCCDFEntity

Represents a security control level entity.

id

The unique identifier for the level.

Type:

str

inherits_from

The identifier of the level from which this level inherits, if any.

Type:

str or None

Returns:

An instance of the Level class.

Return type:

Level

KEYS: Dict[str, Callable[[], Any | None]] = {'id': <function Level.<lambda>>, 'inherits_from': <function Level.<lambda>>}

classmethod from_level_dict(level_dict)

Create an instance of the class from a dictionary.

Parameters:

level_dict (dict) – A dictionary containing the level data. Must include the key “id”. May include the key “inherits_from”.

Returns:

An instance of the class with attributes set according to the dictionary.

class ssg.controls.Policy(filepath, env_yaml=None)

Bases: XCCDFEntity

Represents a security policy defined in an XCCDF file.

id

The identifier of the policy.

Type:

str

env_yaml

The environment YAML configuration.

Type:

dict

filepath

The file path to the policy definition.

Type:

str

controls_dir

The directory containing control definitions.

Type:

str

controls

A list of controls associated with the policy.

Type:

list

controls_by_id

A dictionary mapping control IDs to control objects.

Type:

dict

levels

A list of levels defined for the policy.

Type:

list

levels_by_id

A dictionary mapping level IDs to level objects.

Type:

dict

title

The title of the policy.

Type:

str

source

The source of the policy.

Type:

str

reference_type

The type of reference used in the policy.

Type:

str

product

A list of products associated with the policy.

Type:

list

add_references(rules)

Adds references to the controls based on the provided rules.

This method checks if the reference type is specified and if the current product matches the product specified in the environment YAML. It also verifies if the reference type is allowed based on the reference URIs defined in the environment YAML. If all checks pass, it adds references to the controls by calling the add_references method on each control.

Parameters:

rules (dict) – A dictionary containing the rules to be applied.

Raises:

ValueError – If the reference type is unknown.

check_all_rules_exist(existing_rules)

Checks if all rules in the controls exist in the given set of existing rules.

Parameters:

existing_rules (set) – A set of existing rule identifiers.

Raises:

ValueError – If any control contains rules that do not exist in the given set of existing rules.

check_levels_validity()

Validates the levels defined for each control in the policy.

This function iterates through all controls in the policy and ensures that the levels defined for each control are valid according to the policy’s defined levels. If the policy does not have any levels defined, it implicitly assumes that all controls should have the “default” level.

Raises:

ValueError – If a control has a level that is not defined in the policy’s allowed levels, an error message is raised indicating the invalid level, the control ID, the file path, and the allowed levels.

property default_level

Returns the default level.

If the instance has levels, it returns a list containing the id of the first level. Otherwise, it returns a list containing the string “default”.

Returns:

A list containing either the id of the first level or the string “default”.

Return type:

list

get_control(control_id)

Retrieve a control by its ID.

Parameters:

control_id (str) – The ID of the control to retrieve.

Returns:

The control object associated with the given ID.

Return type:

Control

Raises:

ValueError – If the control ID is not found in the policy.

get_level(level_id)

Retrieve a level by its ID.

Parameters:

level_id (str) – The ID of the level to retrieve.

Returns:

The level object corresponding to the given ID.

Return type:

Level

Raises:

ValueError – If the level ID is not found in the levels_by_id dictionary.

get_level_with_ancestors_sequence(level_id)

Retrieve a sequence of levels starting from the specified level and including all its ancestor levels.

This method uses an OrderedDict to maintain the order of levels and ensure compatibility with Python 2. It starts with the given level and recursively adds all ancestor levels, avoiding duplicates.

Parameters:

level_id (str) – The ID of the starting level.

Returns:

A list of levels starting from the specified level and including all its ancestors.

Return type:

list

load()

Loads and processes the YAML configuration file specified by self.filepath.

This method performs the following steps: 1. Opens and expands the YAML file. 2. Extracts and sets various attributes from the YAML contents, such as controls_dir,

id, policy, title, source, reference_type, and product.

3. Processes the levels section of the YAML file and populates the self.levels and self.levels_by_id attributes.

4. Loads the controls tree from the specified directory or directly from the YAML contents.

5. Saves the controls tree and checks the validity of the levels.

Raises:

KeyError – If required keys are missing in the YAML contents.

remove_selections_not_known(known_rules)

Remove selections and unselections that are not in the known rules.

This method updates the selected and unselected attributes of each control in self.controls by retaining only those elements that are present in the known_rules set.

Parameters:

known_rules (set) – A set of known rule identifiers.

represent_as_dict()

Represents the object as a dictionary.

Returns:

A dictionary representation of the policy object, containing the following keys:

• ”id”: The ID of the object.

• ”policy”: The policy associated with the object.

• ”title”: The title of the object.

• ”source”: The source of the object.

• ”definition_location”: The file path where the object is defined.

• ”controls”: A list of dictionaries representing the controls associated with the object.

• ”levels”: A list of dictionaries representing the levels associated with the object.

Return type:

dict

save_controls_tree(tree)

Saves the controls tree by parsing the given tree and appending each control to the controls list and controls_by_id dictionary.

Parameters:

tree (Any) – The tree structure containing controls to be parsed and saved.

class ssg.controls.Status(status)

Bases: object

The Status class represents various statuses that can be assigned to controls.

SUPPORTED

Indicates the control is supported for the relevant products but is not enabled by default.

Type:

str

PLANNED

Indicates that maintainers plan to work on the control in the near future.

Type:

str

PENDING

Indicates the control is not yet automated and there is no plan to work on it soon.

Type:

str

PARTIAL

Indicates the control is partially automated but requires additional work for full automation.

Type:

str

NOT_APPLICABLE

Indicates the control is not applicable for the products that are being evaluated or targeted by the policy.

Type:

str

MANUAL

Indicates the control requires manual intervention and is not expected to be automated.

Type:

str

INHERENTLY_MET

Indicates the control is inherently met. This means that the control is satisfied by the nature of the system or environment, without requiring additional configuration or actions.

Type:

str

DOES_NOT_MEET

Indicates the control does not meet requirements.

Type:

str

DOCUMENTATION

Indicates the control is related to documentation.

Type:

str

AUTOMATED

Indicates the control is fully automated.

Type:

str

AUTOMATED = 'automated'

DOCUMENTATION = 'documentation'

DOES_NOT_MEET = 'does not meet'

INHERENTLY_MET = 'inherently met'

MANUAL = 'manual'

NOT_APPLICABLE = 'not applicable'

PARTIAL = 'partial'

PENDING = 'pending'

PLANNED = 'planned'

SUPPORTED = 'supported'

classmethod from_control_info(ctrl, status)

Create a control status from the given control information.

Parameters:

• cls (Type) – The class type that this method is called on.

• ctrl (str) – The control identifier.

• status (str or None) – The status of the control.

Returns:

The status of the control if valid, otherwise cls.PENDING.

Return type:

str

Raises:

InvalidStatus – If the given status is not in the list of valid statuses.

classmethod get_status_list()

Retrieve a list of valid status constants.

Returns:

A list containing the valid status constants defined in the class.

Return type:

list

ssg.constants module

class ssg.constants.OvalNamespaces

Bases: object

definition = 'http://oval.mitre.org/XMLSchema/oval-definitions-5'

independent = 'http://oval.mitre.org/XMLSchema/oval-definitions-5#independent'

linux = 'http://oval.mitre.org/XMLSchema/oval-definitions-5#linux'

oval = 'http://oval.mitre.org/XMLSchema/oval-common-5'

class ssg.constants.Reference(id, name, url, regex_with_groups)

Bases: tuple

id

Alias for field number 0

name

Alias for field number 1

regex_with_groups

Alias for field number 3

url

Alias for field number 2

ssg.contributors module

Common functions for processing Contributors in SSG

ssg.contributors.generate()

Generates a list of contributors in both Markdown and XML formats.

This function retrieves the list of contributors from the ‘git shortlog -se’ command, processes the contributions by email, and maps them to contributor names. It then formats the contributors’ information into Markdown and XML strings.

Returns:

A tuple containing two strings:

• contributors_md (str): The contributors list in Markdown format.

• contributors_xml (str): The contributors list in XML format.

Return type:

tuple

ssg.environment module

Common functions for processing Environment in SSG

ssg.environment.open_environment(build_config_yaml_path, product_yaml_path, product_properties_path=None)

Opens and merges environment configurations from given YAML files.

Parameters:

• build_config_yaml_path (str) – The file path to the build configuration YAML file.

• product_yaml_path (str) – The file path to the product YAML file.

• product_properties_path (str, optional) – The directory path containing product properties files. Defaults to None.

Returns:

A dictionary containing the merged contents of the build configuration and product

YAML files.

Return type:

dict

ssg.fixes module

Common functions for processing Fixes in SSG

ssg.fixes.applicable_platforms(fix_path)

Determines the applicable platforms for a given fix.

Parameters:

fix_path (str) – The file path to the fix configuration file.

Returns:

A list of platforms that the fix applies to.

Return type:

list

Raises:

ValueError – If the fix configuration is malformed or missing the ‘platform’ key.

ssg.fixes.find_platform_line(fix_contents)

Parses the fix content to determine the line number that the platforms configuration option is on.

Note

If the configuration specification changes, please update the corresponding parsing in ssg.build_remediations.parse_from_file_with_jinja(…).

Parameters:

fix_contents (list of str) – The contents of the configuration file as a list of strings.

Returns:

The line number of the platform configuration option, or None if not found.

Return type:

int or None

ssg.fixes.get_fix_contents(rule_obj, lang, fix_id)

Retrieve the path and contents of a specific fix.

Parameters:

• rule_obj (object) – The rule object containing the fix information.

• lang (str) – The language of the fix.

• fix_id (str) – The identifier of the fix.

Returns:

A tuple containing the path to the fix and the contents of the fix.

Return type:

tuple

ssg.fixes.get_fix_path(rule_obj, lang, fix_id)

Return the full path to the fix for the given language and fix_id in the rule described by rule_obj.

Parameters:

• rule_obj (dict) – A dictionary containing information about the rule, including dir’, ‘id’, and ‘remediations’.

• lang (str) – The language for which the fix is required.

• fix_id (str) – The identifier for the specific fix.

Returns:

The full path to the fix file.

Return type:

str

Raises:

ValueError – If the rule_obj is malformed or if the fix_id is unknown for the given rule_id and language.

ssg.fixes.set_applicable_platforms(fix_contents, new_platforms)

Modifies the given fix contents to update the platforms to the new platforms.

Parameters:

• fix_contents (list of str) – The contents of the fix file as a list of strings.

• new_platforms (list of str) – A list of new platforms to set in the fix file.

Returns:

The modified contents with the updated platforms.

Return type:

list of str

Raises:

ValueError – If the platform line cannot be found in the fix contents.

ssg.id_translate module

Common functions for processing ID Translations in SSG

class ssg.id_translate.IDTranslator(content_id)

Bases: object

IDTranslator is a class designed to handle the mapping of meaningful, human-readable names to IDs in the formats required by the SCAP checking systems, such as OVAL and OCIL.

content_id

The content identifier used in generating IDs.

Type:

str

generate_id(tagname, name)

Generates a unique identifier string based on the provided tag name and name.

Parameters:

• tagname (str) – The tag name to be used in the identifier.

• name (str) – The name to be used in the identifier.

Returns:

A unique identifier string in the format

”<namespace_prefix>:<content_id>-<name>:<tagname_abbrev>:1”.

Return type:

str

translate(tree, store_defname=False)

Translates the IDs of elements in an XML tree to new identifiers.

Parameters:

• tree (ElementTree.Element) – The XML tree to be processed.

• store_defname (bool, optional) – If True, stores the old name in the metadata for OVAL definitions. Defaults to False.

Returns:

The processed XML tree with updated IDs.

Return type:

ElementTree.Element

The function iterates through each element in the provided XML tree and performs the following actions based on the element’s tag and attributes: - If the element has an “id” attribute, it generates a new ID and sets it. - If store_defname is True and the element is an OVAL definition, it stores the old ID

in the metadata.

• For specific tags like “filter”, “var_ref”, and “object_reference”, it updates the text content with a new ID.

• For attributes that match keys in OVALREFATTR_TO_TAG or OCILREFATTR_TO_TAG, it updates the attribute value with a new ID.

• For the “test_action_ref” tag, it updates the text content with a new ID.

translate_oval_document(oval_document, store_defname=False)

Translates and validates an OVAL document.

This method translates the IDs in the given OVAL document and validates its references.

Parameters:

• oval_document – The OVAL document to be translated and validated.

• store_defname (bool, optional) – If True, stores the definition name during translation. Defaults to False.

Returns:

The translated and validated OVAL document.

ssg.jinja module

Common functions for processing Jinja2 in SSG

class ssg.jinja.AbsolutePathFileSystemLoader(encoding='utf-8')

Bases: BaseLoader

AbsolutePathFileSystemLoader is a custom Jinja2 template loader that loads templates from the file system using absolute paths.

encoding

The encoding used to read the template files. Defaults to ‘utf-8’.

Type:

str

get_source(environment, template)

Retrieves the source code of a Jinja2 template from the file system.

Parameters:

• environment (jinja2.Environment) – The Jinja2 environment.

• template (str) – The absolute path to the template file.

Returns:

A tuple containing the template contents as a string, the template path, and a

function to check if the template file has been updated.

Return type:

tuple

Raises:

• jinja2.TemplateNotFound – If the template file does not exist or the path is not absolute.

• RuntimeError – If there is an error reading the template file.

class ssg.jinja.JinjaEnvironment(bytecode_cache=None)

Bases: Environment

exception ssg.jinja.MacroError

Bases: RuntimeError

ssg.jinja.add_python_functions(substitutions_dict)

Adds predefined Python functions to the provided substitutions dictionary.

The following functions are added: - ‘product_to_name’: Maps a product identifier to its name. - ‘name_to_platform’: Maps a name to its platform. - ‘product_to_platform’: Maps a product identifier to its platform. - ‘url_encode’: Encodes a URL. - ‘raise’: Raises an exception. - ‘expand_yaml_path’: Expands a YAML path.

Parameters:

substitutions_dict (dict) – The dictionary to which the functions will be added.

ssg.jinja.expand_yaml_path(path, parameter)

Expands a dot-separated YAML path into a formatted YAML string.

Parameters:

• path (str) – The dot-separated path to be expanded.

• parameter (str) – An additional parameter to be appended at the end of the path.

Returns:

A formatted YAML string representing the expanded path.

Return type:

str

ssg.jinja.initialize(substitutions_dict)

ssg.jinja.load_macros(substitutions_dict=None)

Augments the provided substitutions_dict with project Jinja macros found in the in JINJA_MACROS_DIRECTORY from constants.py.

Parameters:

substitutions_dict (dict, optional) – A dictionary to be augmented with Jinja macros. Defaults to None.

Returns:

The updated substitutions_dict containing the Jinja macros.

Return type:

dict

ssg.jinja.load_macros_from_content_dir(content_dir, substitutions_dict=None)

Augments the provided substitutions_dict with project Jinja macros found in a specified content directory.

Parameters:

• content_dir (str) – The base directory containing the ‘shared/macros’ subdirectory.

• substitutions_dict (dict, optional) – A dictionary to be augmented with Jinja macros. Defaults to None.

Returns:

The updated substitutions_dict containing the Jinja macros.

Return type:

dict

ssg.jinja.preload_macros(env)

ssg.jinja.process_file(filepath, substitutions_dict)

Process the Jinja file at the given path with the specified substitutions.

Parameters:

• filepath (str) – The path to the Jinja file to be processed.

• substitutions_dict (dict) – A dictionary containing the substitutions to be applied to the template.

Returns:

The rendered template as a string.

Return type:

str

ssg.jinja.raise_exception(message)

ssg.jinja.render_template(data, template_path, output_path, loader)

Renders a template with the given data and writes the output to a file.

Parameters:

• data (dict) – The data to be used in the template rendering.

• template_path (str) – The path to the template file.

• output_path (str) – The path where the rendered output will be written.

• loader (jinja2.BaseLoader) – The Jinja2 loader to use for loading templates.

Returns:

None

ssg.jinja.update_substitutions_dict(filename, substitutions_dict)

Update the substitutions dictionary with macro definitions from a Jinja2 file.

This function treats the given filename as a Jinja2 file containing macro definitions. It exports definitions that do not start with an underscore (_) into the substitutions_dict, which is a dictionary mapping names to macro objects. During the macro compilation process, symbols that already exist in substitutions_dict may be used by the new definitions.

Parameters:

• filename (str) – The path to the Jinja2 file containing macro definitions.

• substitutions_dict (dict) – The dictionary to update with new macro definitions.

Returns:

None

ssg.jinja.url_encode(source)

Encodes a given string into a URL-safe format.

Parameters:

source (str) – The string to be URL-encoded.

Returns:

The URL-encoded string.

Return type:

str

ssg.oval module

Common functions for processing OVAL in SSG

ssg.oval.applicable_platforms(oval_file, oval_version_string=None)

Returns the applicable platforms for a given OVAL file.

This function processes an OVAL file to extract the platforms it applies to. It uses a specified OVAL version string or a default version if none is provided. The function constructs an XML tree from the OVAL file and extracts platform information from it.

Parameters:

• oval_file (str) – The path to the OVAL file to be processed.

• oval_version_string (str, optional) – The OVAL version string to be used. If not provided, a default version is used.

Returns:

A list of platforms that the OVAL file applies to.

Return type:

list

Raises:

Exception – If there is an error while parsing the OVAL file.

ssg.oval.parse_affected(oval_contents)

Returns the tuple (start_affected, end_affected, platform_indents) for the passed OVAL file contents.

Args: oval_contents (list of str): The contents of the OVAL file, where each element is a line from

the file.

Returns:

A tuple containing:

• start_affected (int): The line number of the starting <affected> tag.

• end_affected (int): The line number of the closing </affected> tag.

• platform_indents (str): The indenting characters before the contents of the <affected> element.

Return type:

tuple

Raises:

ValueError – If the OVAL file does not contain a single <affected> element, if the start tag is after the end tag, or if the tags contain other elements.

ssg.parse_oval module

Common functions for OVAL parsing in SSG

class ssg.parse_oval.ElementFinder(oval_groups)

Bases: object

A class to find specific elements within an XML structure based on given criteria.

oval_groups

A dictionary containing groups of OVAL definitions.

Type:

dict

target

The name of the target element to find.

Type:

str

attrib

The attribute of the target element to retrieve.

Type:

str

result

A set to store the results of the found elements’ attributes.

Type:

set

find_element(start_element, target_element_name, sought_attrib)

Find elements in an XML tree that match the target element name and sought attribute.

Parameters:

• start_element (Element) – The starting element to begin the search from.

• target_element_name (str) – The name of the target element to find.

• sought_attrib (str) – The attribute to look for in the target elements.

Returns:

This method does not return a value. The results are stored in the instance

variable self.result.

Return type:

None

ssg.parse_oval.find_extending_defs(oval_groups, defn)

Find and return the definitions that extend a given definition.

Parameters:

• oval_groups (dict) – A dictionary containing OVAL groups.

• defn (str) – The definition to find extensions for.

Returns:

A list of definitions that extend the given definition.

Return type:

list

ssg.parse_oval.get_container_groups(fname)

Parses an OVAL file and retrieves container groups.

Parameters:

fname (str) – The path to the OVAL file to be parsed.

Returns:

A list of container groups extracted from the OVAL file.

Return type:

list

ssg.parse_oval.resolve_definition(oval_groups, defn)

Resolves a definition by finding the attribute ‘external_variable’ with the specified ‘id’ in the given OVAL groups.

Parameters:

• oval_groups (dict) – A dictionary containing OVAL groups.

• defn (str) – The definition to resolve.

Returns:

The value of the ‘external_variable’ attribute with the specified ‘id’ if found, otherwise None.

ssg.playbook_builder module

class ssg.playbook_builder.PlaybookBuilder(product_yaml_path, input_dir, output_dir, rules_dir, profiles_dir, build_config_yaml)

Bases: object

build(profile_id=None, rule_id=None)

Creates Playbooks for a specified profile. If profile is not given, creates playbooks for all profiles in the product. If the rule_id is not given, Playbooks are created for every rule.

choose_variable_value(var_id, variables, refinements)

Determine value of variable based on profile refinements.

create_playbook(snippet_path, rule_id, variables, refinements, output_dir)

Creates a Playbook from Ansible snippet for the given rule specified by rule ID, fills in the profile values and saves it into output_dir.

create_playbook_for_single_rule(profile, rule_id, variables)

Creates a Playbook for given rule specified by a rule_id. Created Playbooks are parametrized by variables according to profile selection. Playbooks are written into a new subdirectory in output_dir.

create_playbooks_for_all_rules(variables)

create_playbooks_for_all_rules_in_profile(profile, variables)

Creates a Playbook for each rule selected in a profile from tasks extracted from snippets. Created Playbooks are parametrized by variables according to profile selection. Playbooks are written into a new subdirectory in output_dir.

get_benchmark_variables()

Get all variables, their selectors and values used in a given benchmark. Returns a dictionary where keys are variable IDs and values are dictionaries where keys are selectors and values are variable values.

get_data_from_snippet(snippet_yaml, variables, refinements)

Extracts and resolves tasks and variables from Ansible snippet.

open_profile(profile_path)

Opens and parses profile at the given profile_path.

ssg.products module

Common functions for processing Products in SSG

class ssg.products.Product(filename)

Bases: object

A class to represent a product with primary and acquired data.

_primary_data

A dictionary to store primary data loaded from a file.

Type:

dict

_acquired_data

A dictionary to store additional data acquired after initialization.

Type:

dict

expand_by_acquired_data(property_dict)

Expands the current object with properties from the given dictionary.

This method updates the object’s acquired data with the properties provided in the property_dict. If any property in property_dict already exists in the object, a ValueError is raised.

Parameters:

property_dict (dict) – A dictionary containing properties to be added.

Returns:

None

Raises:

ValueError – If any property in property_dict is already defined in the object.

get(key, default=None)

Retrieve the value associated with the given key from the internal data dictionary.

Parameters:

• key (str) – The key to look up in the dictionary.

• default (optional) – The value to return if the key is not found. Defaults to None.

Returns:

The value associated with the key if it exists, otherwise the default value.

read_properties_from_directory(path)

Reads YAML property files from the specified directory, processes them, and updates the current object with the new data.

Parameters:

path (str) – The directory path containing the YAML files.

Returns:

None

static transform_default_and_overrides_mappings_to_mapping(mappings)

Transforms a dictionary containing ‘default’ and ‘overrides’ mappings into a single mapping.

This function expects a dictionary with at least a ‘default’ key and optionally an ‘overrides’ key. It merges the ‘default’ mapping with the ‘overrides’ mapping, with ‘overrides’ taking precedence in case of key conflicts. If the input dictionary contains any other keys, a ValueError is raised.

Parameters:

mappings (dict) – A dictionary containing ‘default’ and optionally ‘overrides’ mappings.

Returns:

A merged dictionary of ‘default’ and ‘overrides’ mappings.

Return type:

dict

Raises:

ValueError – If the input is not a dictionary, if the ‘default’ key is missing, or if there are any keys other than ‘default’ and ‘overrides’ in the input dictionary.

write(filename)

Writes the data to a specified file in an ordered format.

Parameters:

filename (str) – The name of the file to write the data to.

Returns:

None

Raises:

IOError – If the file cannot be opened or written to.

ssg.products.get_all(ssg_root)

Analyzes all products in the SSG root and sorts them into two categories.

Those which use linux_os and those which use their own directory.

Parameters:

ssg_root (str) – The root directory of the SSG.

Returns:

A namedtuple containing two sets:

• linux (set): A set of products that use linux_os.

• other (set): A set of products that use their own directory.

Return type:

namedtuple

ssg.products.get_all_product_yamls(ssg_root)

Generator function that yields product names and their corresponding YAML data.

Parameters:

ssg_root (str) – The root directory where the product directories are located.

Yields:

tuple – A tuple containing the product name and its corresponding YAML data.

ssg.products.get_all_products_with_same_guide_directory(ssg_root, product_yaml)

Generator function that yields product YAMLs of all products that share the same guide directory.

Parameters:

• ssg_root (str) – The root directory of the SSG.

• product_yaml (dict) – The YAML data of the current product, containing at least ‘product_dir’, ‘benchmark_root’, and ‘product’ keys.

Yields:

dict – The YAML data of products that have the same guide directory but different product IDs.

ssg.products.get_profile_files_from_root(env_yaml, product_yaml)

Retrieves a list of profile files from the specified root directory.

Parameters:

• env_yaml (dict) – A dictionary containing environment configuration.

• product_yaml (dict) – A dictionary containing product configuration, including the base directory under the key “product_dir”.

Returns:

A sorted list of profile file paths found in the profiles directory.

Return type:

list

ssg.products.get_profiles_directory(env_yaml)

Retrieves the profiles directory path from the given environment configuration.

Parameters:

env_yaml (dict) – A dictionary containing environment configuration.

Returns:

The path to the profiles directory if found, otherwise None.

Return type:

str

ssg.products.load_product_yaml(product_yaml_path: str) → Product

Reads a product data from disk and returns it.

The returned product dictionary also contains derived useful information.

Parameters:

product_yaml_path (str) – The file path to the product YAML file.

Returns:

A dictionary containing the product data and derived useful information.

Return type:

dict

ssg.products.product_yaml_path(ssg_root, product)

Constructs the file path to the product YAML file.

Parameters:

• ssg_root (str) – The root directory of the SSG.

• product (str) – The name of the product.

Returns:

The full file path to the product YAML file.

Return type:

str

ssg.profiles module

class ssg.profiles.ProfileSelections(profile_id, profile_title, product_id, product_title)

Bases: object

A class to represent profile with sections of rules and variables.

Attributes:

profile_idstr

The unique identifier for the profile.

profile_titlestr

The profile title associated with the profile id.

product_idstr

The product id associated with the profile.

product_titlestr

The product title associated with the product id.

ssg.profiles.get_profiles_from_products(content_dir: str, products: list, sorted: bool = False) → list

Retrieves profiles with respective variables from the given products.

Parameters:

• content_dir (str) – The directory containing the content.

• products (list) – A list of product names to retrieve profiles from.

• sorted (bool) – Sorts the profile selections if true, defaults to False

Returns:

A list of ProfileVariables objects containing profile variables for each product.

Return type:

list

ssg.rule_dir_stats module

This module contains common code shared by utils/rule_dir_stats.py and utils/rule_dir_diff.py. This code includes functions for walking the output of the utils/rule_dir_json.py script, and filtering functions used in both scripts.

ssg.rule_dir_stats.filter_rule_ids(all_keys, queries)

Filters a set of keys based on a set of queries.

A set of queries is a comma separated list of queries, where a query is either a rule id or a substring thereof.

Parameters:

• all_keys (iterable) – An iterable containing all possible keys.

• queries (str) – A comma-separated list of queries, where each query is either a rule id or a substring thereof. If the literal string “all” is provided, all keys are returned.

Returns:

A set of keys from all_keys that match any of the queries. If queries is empty, an

empty set is returned. If queries is “all”, all keys are returned.

Return type:

set

ssg.rule_dir_stats.get_affected_products(rule_obj)

Extracts and returns the set of affected products from a given rule object.

Parameters:

rule_obj (dict) – A dictionary representing a rule, which contains a ‘products’ key.

Returns:

A set of products affected by the rule.

Return type:

set

ssg.rule_dir_stats.get_all_affected_products(args, rule_obj)

From a rule_obj, return the set of affected products from rule.yml, and all fixes and checks.

If args.strict is set, this function is equivalent to get_affected_products. Otherwise, it includes ovals and fix content based on the values of args.fixes_only and args.ovals_only.

Parameters:

• args (Namespace) – The arguments passed to the script, containing flags such as strict, fixes_only, and ovals_only.

• rule_obj (dict) – The rule object containing information about the rule, including affected products, oval products, and remediation products.

Returns:

A set of affected products based on the rule object and the provided arguments.

Return type:

set

ssg.rule_dir_stats.missing_oval(rule_obj)

For a rule object, check if it is missing an OVAL.

Parameters:

rule_obj (dict) – A dictionary representing the rule object. It must contain the keys ‘id’ and ‘ovals’.

Returns:

A message indicating the rule ID that is missing all OVALs, or None if the rule has

OVALs.

Return type:

str

ssg.rule_dir_stats.missing_remediation(rule_obj, r_type)

Check if a rule object is missing a remediation of a specified type.

Parameters:

• rule_obj (dict) – The rule object containing rule details.

• r_type (str) – The type of remediation to check for.

Returns:

A message indicating the rule ID and the missing remediation type, if the remediation

is missing. Otherwise, returns None.

Return type:

str

ssg.rule_dir_stats.product_names_oval(rule_obj)

Checks the consistency between the product names and OVAL object names for a given rule object.

Parameters:

rule_obj (dict) –

A dictionary representing a rule object. It should contain: - ‘id’ (str): The identifier of the rule. - ‘ovals’ (dict): A dictionary where keys are OVAL filenames and values are

dictionaries containing ‘products’ (list of product names).

Returns:

A message indicating if there is a mismatch between the product name and OVAL object

name. The message format is “rule_id:<rule_id> has a different product and OVALs names: <product> is not <oval_product>”. Returns None if all product names match their corresponding OVAL object names.

Return type:

str

ssg.rule_dir_stats.product_names_remediation(rule_obj, r_type)

Checks the consistency between the scope of platforms and the product names of the remediations of a given type for a rule object.

Parameters:

• rule_obj (dict) – A dictionary representing the rule object, which contains an ‘id’ key and a ‘remediations’ key. The ‘remediations’ key is a dictionary where keys are remediation types and values are dictionaries of remediation names and their associated products.

• r_type (str) – The type of remediation to check (e.g., ‘bash’, ‘ansible’).

Returns:

A message indicating the rule ID and the inconsistency found, if any.

The message specifies the rule ID, the remediation type, the product name that is inconsistent, and the expected product name. Returns None if no inconsistencies are found.

Return type:

str

ssg.rule_dir_stats.two_plus_oval(rule_obj)

Check if a rule object has two or more OVALs.

Parameters:

rule_obj (dict) – A dictionary representing a rule object. It should have the following keys: - ‘id’ (str): The identifier of the rule. - ‘ovals’ (list): A list of OVAL identifiers associated with the rule.

Returns:

A formatted string indicating the rule ID and its associated OVALs if there are two

or more OVALs.

Return type:

str

ssg.rule_dir_stats.two_plus_remediation(rule_obj, r_type)

Check if a rule object has two or more remediations of a specified type.

Parameters:

• rule_obj (dict) – The rule object containing rule details and remediations.

• r_type (str) – The type of remediation to check for.

Returns:

A formatted string indicating the rule ID and the remediations if there are two or

more of the specified type.

Return type:

str

ssg.rule_dir_stats.walk_rule_stats(rule_output)

Walk the output of a rule, generating statistics about affected ovals, remediations, and generating verbose output in a stable order.

Parameters:

rule_output (dict) – The output of a rule containing information about ovals and remediations.

Returns:

A tuple containing the following elements:

• affected_ovals (int): The number of affected ovals.

• affected_remediations (int): The number of affected remediations.

• all_affected_remediations (int): The number of rules where all remediations are affected.

• affected_remediations_type (defaultdict): A dictionary with the count of each type of affected remediation.

• all_output (list): A list of all affected ovals and remediations in a stable order.

Return type:

tuple

ssg.rule_dir_stats.walk_rules(args, known_rules, oval_func, remediation_func)

Walks through a dictionary of known rules, conditionally calling provided functions to generate OVAL and remediation content, and returns the number of visited rules along with the output for each visited rule.

Parameters:

• args (object) – An object containing arguments that control the behavior of the function.

• known_rules (dict) – A dictionary where keys are rule IDs and values are rule objects.

• oval_func (function) – A function to be called for each rule to generate OVAL content.

• remediation_func (function) – A function to be called for each rule to generate remediation content.

Returns:

affected_rules (int): The number of rules that were visited. verbose_output (dict): A dictionary containing the output for each visited rule.

Return type:

tuple

The output structure is a dict as follows:

{
    rule_id: {
        "oval": oval_func(args, rule_obj),
        "ansible": remediation_func(args, "ansible", rule_obj),
        "anaconda": remediation_func(args, "anaconda", rule_obj),
        "bash": remediation_func(args, "bash", rule_obj),
        "puppet": remediation_func(args, "puppet", rule_obj)
    },
    ...
}

The arguments supplied to oval_func are args and rule_obj. The arguments supplied to remediation_func are args, the remediation type, and rule_obj. The input rule_obj structure is the value of known_rules[rule_id].

ssg.rule_dir_stats.walk_rules_diff(args, left_rules, right_rules, oval_func, remediation_func)

Walk through two dictionaries of known rules and generate five sets of output.

Does not understand renaming of rule_ids as this would depend on disk content to reflect these differences. Unless significantly more data is added to the rule_obj structure (contents of rule.yml, ovals, remediations, etc.), all information besides ‘title’ is not uniquely identifying or could be easily updated.

Parameters:

• args – Arguments to be passed to the walk_rules and walk_rules_parallel functions.

• left_rules (dict) – Dictionary of rules on the left side.

• right_rules (dict) – Dictionary of rules on the right side.

• oval_func (function) – Function to process OVAL definitions.

• remediation_func (function) – Function to process remediation scripts.

Returns:

A five-tuple containing:

• left_only_data: Data for rules only in left_rules.

• right_only_data: Data for rules only in right_rules.

• left_changed_data: Data for rules in both left_rules and right_rules but changed in left_rules.

• right_changed_data: Data for rules in both left_rules and right_rules but changed in right_rules.

• common_data: Data for rules common to both left_rules and right_rules.

Return type:

tuple

ssg.rule_dir_stats.walk_rules_diff_stats(results)

Takes the results of walk_rules_diff (results) and generates five sets of output statistics.

Parameters:

results (list) – A list of five elements, where each element is a tuple containing affected rules and verbose output.

Returns:

A tuple containing five elements, each representing the statistics for left_only

rules, right_only rules, shared left rules, shared right rules, and shared common rules. Each element in the tuple is itself a tuple containing: - affected_rules (int): Number of affected rules. - affected_ovals (int): Number of affected OVAL definitions. - affected_remediations (int): Number of affected remediations. - all_affected_remediations (int): Total number of affected remediations. - affected_remediations_type (dict): Dictionary with remediation types as keys and

counts as values.

• all_output (list): List of all output data.

Return type:

tuple

Raises:

AssertionError – If the length of results is not 5 or the length of output_data is not 5.

ssg.rule_dir_stats.walk_rules_parallel(args, left_rules, right_rules, oval_func, remediation_func)

Walks two sets of known_rules (left_rules and right_rules) with identical keys and returns left_only, right_only, and common_only output from _walk_rule.

If the outputted data for a rule when called on left_rules and right_rules is the same, it is added to common_only. Only rules which output different data will have their data added to left_only and right_only respectively.

Parameters:

• args – Arguments to be passed to the _walk_rule function.

• left_rules (dict) – Dictionary of rules on the left side.

• right_rules (dict) – Dictionary of rules on the right side.

• oval_func (function) – Function to process OVAL definitions.

• remediation_func (function) – Function to process remediation scripts.

Returns:

A tuple containing three elements:

• left_only (tuple): A tuple containing the count of affected rules and the verbose output for rules only in left_rules.

• right_only (tuple): A tuple containing the count of affected rules and the verbose output for rules only in right_rules.

• common_only (tuple): A tuple containing the count of affected rules and the verbose output for rules common to both left_rules and right_rules.

Return type:

tuple

Raises:

AssertionError – If the sets of keys in left_rules and right_rules are not identical.

ssg.rule_dir_stats.walk_rules_stats(args, known_rules, oval_func, remediation_func)

Walk a dictionary of known_rules and generate simple aggregate statistics for all visited rules.

The oval_func and remediation_func arguments behave according to walk_rules(). An effort is made to provide consistently ordered verbose_output by sorting all visited keys and the keys of ssg.build_remediations.REMEDIATION_MAP.

Parameters:

• args – Arguments passed to the function.

• known_rules (dict) – A dictionary of known rules to be processed.

• oval_func (function) – Function to process OVAL definitions.

• remediation_func (function) – Function to process remediations.

Returns:

A tuple containing:

• affected_rules (int): Number of affected rules.

• affected_ovals (int): Number of affected OVAL definitions.

• affected_remediations (int): Number of affected remediations.

• all_affected_remediations (int): Total number of affected remediations.

• affected_remediations_type (dict): Dictionary with the count of each type of affected remediation.

• all_output (list): Ordered output of all functions.

Return type:

tuple

ssg.rule_yaml module

The rule_yaml module provides various utility functions for handling YAML files containing Jinja macros, without having to parse the macros.

ssg.rule_yaml.add_key_value(contents, key, start_line, new_value)

Adds a new key-value pair to the contents at a specified line.

Does not modify the value of contents.

Parameters:

• contents (list of str) – The original list of strings representing the contents.

• key (str) – The key to be added.

• start_line (int) – The line number after which the new key-value pair should be added.

• new_value (str) – The value associated with the key.

Returns:

A new list of strings with the key-value pair added and a blank line afterwards.

Return type:

list of str

ssg.rule_yaml.find_section_lines(file_contents, sec)

Parses the given file_contents as YAML to find the section with the given identifier.

Note that this does not call into the yaml library and thus correctly handles Jinja macros at the expense of not being a strictly valid yaml parsing.

Parameters:

• file_contents (list of str) – The contents of the file, split into lines.

• sec (str) – The identifier of the section to find.

Returns:

A list of namedtuples (start, end) representing the lines where the

section exists.

Return type:

list of namedtuple

ssg.rule_yaml.get_section_lines(file_path, file_contents, key_name)

From the given file_path and file_contents, find the lines describing the section key_name and returns the line range of the section.

Parameters:

• file_path (str) – The path to the file being analyzed.

• file_contents (str) – The contents of the file as a string.

• key_name (str) – The name of the section to find within the file contents.

Returns:

A tuple representing the start and end line numbers of the section if found. None: If the section is not found.

Return type:

tuple

Raises:

ValueError – If multiple instances of the section are found in the file.

ssg.rule_yaml.get_yaml_contents(rule_obj)

From a rule_obj description, return a namedtuple of (path, contents).

Parameters:

rule_obj (dict) – A dictionary containing information about the rule. It must have the keys ‘dir’ and ‘id’.

Returns:

A namedtuple with ‘path’ as the path to the rule YAML file and ‘contents’ as

the list of lines in the file.

Return type:

namedtuple

Raises:

ValueError – If the YAML file does not exist for the given rule_id.

ssg.rule_yaml.has_duplicated_subkeys(file_path, file_contents, sections)

Checks whether a section has duplicated keys in a YAML file.

Note that these duplicated keys are silently ignored by the YAML parser used.

Parameters:

• file_path (str) – The path to the YAML file.

• file_contents (list of str) – The contents of the YAML file as a list of lines.

• sections (str or list of str) – The section or list of sections to check for duplicated keys.

Returns:

True if any section has duplicated keys, False otherwise.

Return type:

bool

ssg.rule_yaml.parse_from_yaml(file_contents, lines)

Parse the given line range as a YAML, returning the parsed object.

Parameters:

• file_contents (list of str) – The contents of the file as a list of strings.

• lines (slice) – A slice object indicating the start and end lines to parse.

Returns:

The parsed YAML object.

Return type:

object

ssg.rule_yaml.remove_lines(contents, lines)

Remove the specified lines from the contents.

This function takes the contents of a file and a range of lines to be removed, and returns the new contents with those lines removed. The original contents are not modified.

Parameters:

• contents (list of str) – The contents of the file as a list of lines.

• lines (slice) – A slice object representing the range of lines to be removed.

Returns:

The new contents with the specified lines removed.

Return type:

list of str

ssg.rule_yaml.sort_section_keys(file_path, file_contents, sections, sort_func=None)

Sort subkeys in a YAML file’s section.

Parameters:

• file_path (str) – The path to the YAML file.

• file_contents (list of str) – The contents of the YAML file as a list of lines.

• sections (str or list of str) – The section or sections whose subkeys need to be sorted.

• sort_func (callable, optional) – A function to determine the sort order of the subkeys. If None, the subkeys are sorted in ascending order.

Returns:

The modified contents of the YAML file with sorted subkeys in the specified

sections.

Return type:

list of str

Raises:

• ValueError – If a duplicated key is found within the same section.

• AssertionError – If the section contains more than one parent key or if a subkey line cannot be found.

ssg.rule_yaml.update_key_value(contents, key, old_value, new_value)

Find a key in the contents of a file and replace its value with a new value, returning the resulting file contents.

This function validates that the old value is constant and hasn’t changed since parsing its value. Does not modify the original contents.

Parameters:

• contents (list of str) – The contents of the file as a list of strings.

• key (str) – The key whose value needs to be updated.

• old_value (str) – The current value associated with the key.

• new_value (str) – The new value to replace the old value.

Returns:

The updated contents of the file.

Return type:

list of str

Raises:

ValueError – If the key cannot be found in the given contents or if the old value does not match the current value associated with the key.

ssg.rules module

Common functions for processing rules in SSG

ssg.rules.applies_to_product(file_name, product)

Determines if a given file applies to a specified product.

An OVAL or fix is considered applicable to a product if any of the following conditions are met: - The product parameter is Falsy (e.g., None, False, or an empty string). - The file_name is “shared”. - The file_name matches the product. - The product starts with the file_name.

Note that this function only filters based on the file name and does not consider the contents of the fix or check.

Parameters:

• file_name (str) – The name of the file to check.

• product (str) – The product to check against.

Returns:

True if the file applies to the product, False otherwise.

Return type:

bool

ssg.rules.find_rule_dirs(base_dir)

Generator which yields all rule directories within a given base_dir, recursively.

Parameters:

base_dir (str) – The base directory to start searching for rule directories.

Yields:

str – The path to each rule directory found within the base directory.

ssg.rules.find_rule_dirs_in_paths(base_dirs)

Generator which yields all rule directories within a given directories list, recursively.

Parameters:

base_dirs (list) – A list of base directories to search for rule directories.

Yields:

str – Paths to rule directories found within the base directories.

ssg.rules.get_rule_dir_id(path)

Returns the base name of a rule directory.

This function takes a file path and returns the base name of the directory. It correctly handles being passed either the directory path or the YAML metadata file path (ending with ‘rule.yml’).

Parameters:

path (str) – The file or directory path.

Returns:

The base name of the rule directory.

Return type:

str

ssg.rules.get_rule_dir_ovals(dir_path, product=None)

Gets a list of OVALs contained in a rule directory.

If product is None, returns all OVALs. Only returns OVALs which exist.

Parameters:

• dir_path (str) – The path to the rule directory.

• product (str, optional) – The product name to filter OVALs. Defaults to None.

Returns:

A list of paths to OVAL files in the specified directory, ordered by priority.

Return type:

list

ssg.rules.get_rule_dir_sces(dir_path, product=None)

Get a list of SCEs contained in a rule directory.

Only returns SCEs which exist.

Parameters:

• dir_path (str) – The path to the rule directory.

• product (str, optional) – The product name to filter SCEs. If None, returns all SCEs.

Returns:

A list of paths to applicable SCE files. If product is specified, returns SCEs

in the order of priority: - {product}.{ext} - shared.{ext}

Return type:

list

The function performs the following steps:

1. Checks if the provided directory is a valid rule directory.

2. Checks if the “sce” subdirectory exists within the rule directory.

3. Iterates over the files in the “sce” directory, filtering and prioritizing them based on the product.

4. Returns a list of applicable SCE file paths, with product-specific SCEs listed before shared SCEs.

ssg.rules.get_rule_dir_yaml(dir_path)

Constructs the path to the YAML metadata file for a given rule directory, regardless of if it exists.

Parameters:

dir_path (str) – The path to the rule directory.

Returns:

The path to the “rule.yml” file within the specified directory.

Return type:

str

ssg.rules.is_rule_dir(dir_path)

Check if a given directory path is a valid rule directory.

A valid rule directory must: 1. Exist as a directory. 2. Contain a specific YAML file as determined by get_rule_dir_yaml().

Parameters:

dir_path (str) – The path to the directory to check.

Returns:

True if dir_path is a valid rule directory, False otherwise.

Return type:

bool

ssg.requirement_specs module

Common functions for processing Requirements Specs in SSG

class ssg.requirement_specs.Requirement(obj: str)

Bases: object

A class to represent a package requirement with version specifications.

_req

The parsed requirement object.

Type:

RequirementParser

_specs

The set of version specifiers for the requirement.

Type:

utils.VersionSpecifierSet

property arg

Retrieve the first extra requirement if available.

Returns:

The first element in the extras list if it exists, otherwise None.

static get_base_for_parametrized(name)

Extracts the base project name from a given parameterized package name.

Parameters:

name (str) – The parameterized package name to parse.

Returns:

The base project name of the package.

Return type:

str

has_version_specs()

Check if the requirement has version specifications.

Returns:

True if there are version specifications, False otherwise.

Return type:

bool

static is_parametrized(name)

Check if a package requirement is parametrized.

A parametrized package requirement includes extras, which are additional features or dependencies that can be optionally included.

Parameters:

name (str) – The name of the package requirement to check.

Returns:

True if the package requirement is parametrized (includes extras),

False otherwise.

Return type:

bool

property name

Retrieve the project name from the requirement specifications.

Returns:

The name of the project.

Return type:

str

property ver_specs

Retrieve the specifications.

Returns:

The specifications stored in the instance.

Return type:

dict

class ssg.requirement_specs.RequirementParser(target_v: str)

Bases: object

A simple parser for package requirements with version specifiers. Handles formats like: package[extra]>=1.0,<2.0

property extras: List[str]

property project_name: str

property specs: List[Tuple[str, str]]

ssg.requirement_specs.parse_version_into_evr(version)

Parses a version string into its epoch, version, and release components.

Parameters:

version (str) – The version string to parse. It should be in the format ‘epoch:version-release’, where ‘epoch’ and ‘release’ are optional.

Returns:

A dictionary with keys ‘epoch’, ‘version’, and ‘release’, containing the respective

parts of the version string. If ‘epoch’ or ‘release’ are not present in the input string, their values will be None.

Return type:

dict

Raises:

ValueError – If the version string does not match the expected format.

ssg.shims module

ssg.templates module

Common functions for processing Templates in SSG

class ssg.templates.Builder(env_yaml, resolved_rules_dir, templates_dir, remediations_dir, checks_dir, platforms_dir, cpe_items_dir)

Bases: object

Class for building all templated content for a given product.

To generate content from templates, pass the env_yaml, path to the directory with resolved rule YAMLs, path to the directory that contains templates, path to the output directory for checks and a path to the output directory for remediations into the constructor. Then, call the method build() to perform a build.

env_yaml

Environment YAML configuration.

Type:

dict

resolved_rules_dir

Path to the directory with resolved rule YAMLs.

Type:

str

templates_dir

Path to the directory that contains templates.

Type:

str

remediations_dir

Path to the output directory for remediations.

Type:

str

checks_dir

Path to the output directory for checks.

Type:

str

platforms_dir

Path to the directory for platforms.

Type:

str

cpe_items_dir

Path to the directory for CPE items.

Type:

str

output_dirs

Dictionary of output directories for different languages.

Type:

dict

templates

Dictionary of loaded templates.

Type:

dict

product_cpes

Instance of ProductCPEs for managing CPE items.

Type:

ProductCPEs

build()

Builds all templated content for all languages and writes the output to the correct build directories.

This method performs the following steps: 1. Creates the necessary output directories. 2. Builds extra OVAL definitions. 3. Builds all rules. 4. Builds all platforms.

Returns:

None

Raises:

OSError – If there is an issue creating the output directories.

build_all_platforms()

Builds all platforms by iterating through the files in the platforms directory.

This method reads each platform file, constructs a Platform object from the YAML data, and then builds the platform using the build_platform method. The platforms are processed in sorted order based on their filenames.

Returns:

None

Raises:

• OSError – If there is an issue reading the platforms directory or files.

• ssg.build_yaml.PlatformError – If there is an issue constructing the Platform object.

build_all_rules()

Builds all rules from YAML files located in the resolved rules directory.

This method iterates over all files in the resolved rules directory, sorts them, and attempts to create a Rule object from each file. If a rule is marked as “documentation-incomplete” and the build is not in debug mode, it skips that rule. If a rule is templated, it calls the build_rule method to process it.

Returns:

None

Raises:

ssg.build_yaml.DocumentationNotComplete – If a rule’s documentation is incomplete and the build is not in debug mode.

build_cpe(cpe)

Builds and processes a CPE (Common Platform Enumeration) item.

This method generates language-specific templates for the given CPE item, processes them based on their template type, and then adds the CPE item to the product’s CPE list. Finally, it dumps the CPE item to a YAML file.

Parameters:

cpe (CPE) – The CPE item to be processed.

Returns:

None

build_extra_ovals()

Builds and processes extra OVAL definitions from a YAML declaration file.

This method reads a YAML file named “extra_ovals.yml” located in the templates directory. It iterates over the items in the YAML file, where each item represents an OVAL definition. For each OVAL definition, it creates a rule instance using the definition ID and template, builds the corresponding language-specific content, and writes the content to the appropriate location.

Returns:

None

Raises:

• FileNotFoundError – If the “extra_ovals.yml” file does not exist in the templates directory.

• ssg.yaml.YAMLError – If there is an error parsing the YAML file.

build_lang_for_templatable(templatable, lang)

Builds templated content of a given Templatable for a selected language.

Parameters:

• templatable (Templatable) – The object that contains the template to be filled.

• lang (str) – The language code for which the template should be filled.

Returns:

The filled template content for the specified language.

Return type:

str

build_platform(platform)

Builds templated content for a given platform, processing all CPEs/Symbols for all available languages.

The method writes the output to the appropriate build directories and updates the platform itself.

Parameters:

platform (Platform) – The platform object containing CPEs/Symbols to be processed.

The method performs the following steps: 1. Identifies languages affecting the platform by examining the symbols associated with the platform’s tests. 2. For each CPE that is templated, it builds the CPE and determines the languages that need to be generated. 3. Updates the platform’s conditional items based on the resolved languages and CPE items. 4. Dumps the platform’s data into a YAML file in the designated platforms directory.

build_rule(rule)

Builds templated content of a given Rule for all available languages, writing the output to the correct build directories.

Parameters:

rule (Rule) – The rule object containing the data to be templated.

Returns:

None

get_lang_contents_for_templatable(templatable, language)

Generate and return the content for a specified language for a given Templatable object.

Parameters:

• templatable (Templatable) – The Templatable object for which to generate content.

• language (Language) – The language for which to generate content.

Returns:

The generated content for the specified language.

Return type:

str

Raises:

RuntimeError – If there is an error generating the template language content.

get_resolved_langs_to_generate(templatable)

Determine the languages to generate for a given Templatable instance.

This method calculates the intersection of the languages supported by the template and the languages specified in the Templatable’s template configuration ‘backends’.

Parameters:

templatable (Templatable) – An instance of a Templatable object.

Returns:

A set of languages that are both supported by the template and specified in the

Templatable’s configuration.

Return type:

set

Raises:

ValueError – If the template used by the Templatable does not exist.

process_template_lang_file(template_name, template_vars, lang, local_env_yaml)

Processes a template for a given template name and language, and returns the rendered content.

Parameters:

• template_name (str) – The name of the template to process.

• template_vars (dict) – A dictionary of variables to be used in the template.

• lang (object) – An object representing the language, which should have a ‘name’ attribute.

• local_env_yaml (dict) – A dictionary of local environment variables to be merged with the global environment variables.

Returns:

The rendered content of the template.

Return type:

str

Raises:

ValueError – If the specified language is not available for the given template.

write_lang_contents_for_templatable(filled_template, lang, templatable)

Writes the filled template content to a file specific to the given language.

Parameters:

• filled_template (str) – The content to be written to the file.

• lang (Language) – The language object containing language-specific details.

• templatable (Templatable) – The templatable object containing the id_ attribute.

The output file name is generated by appending the language-specific file extension to the templatable’s id_. The file is then written to the appropriate output directory for the given language.

class ssg.templates.Template(templates_root_directory, name)

Bases: object

A class to represent a template for content generation.

templates_root_directory

The root directory where templates are stored.

Type:

str

name

The name of the template.

Type:

str

langs

A list to store supported languages.

Type:

list

template_path

The path to the template directory.

Type:

str

template_yaml_path

The path to the template’s YAML configuration file.

Type:

str

preprocessing_file_path

The path to the template’s preprocessing file.

Type:

str

classmethod load_template(templates_root_directory, name)

Load a template if it exists and looks like a template.

Parameters:

• cls – The class that this method belongs to.

• templates_root_directory (str) – The root directory where templates are stored.

• name (str) – The name of the template to load.

Returns:

An instance of the template if it exists and looks like a template, otherwise None.

preprocess(parameters, lang)

Preprocess the given parameters by applying template module preprocessing and converting parameter keys to uppercase.

Parameters:

• parameters (dict) – The parameters to preprocess.

• lang (str) – The language code.

Returns:

The preprocessed parameters with keys in uppercase.

Return type:

dict

class ssg.templates.TemplatingLang(name, file_extension, template_type, lang_specific_dir)

Bases: tuple

file_extension

Alias for field number 1

lang_specific_dir

Alias for field number 3

name

Alias for field number 0

template_type

Alias for field number 2

ssg.templates.load_module(module_name: str, module_path: str)

Loads a Python module from a given file path.

This function attempts to load a module using importlib.

Parameters:

• module_name (str) – The name to assign to the loaded module.

• module_path (str) – The file path to the module to be loaded.

Returns:

The loaded module object.

Return type:

module

Raises:

ValueError – If the module cannot be loaded due to an invalid spec or loader.

ssg.utils module

Utils functions consumed by SSG

exception ssg.utils.SSGError

Bases: RuntimeError

class ssg.utils.VersionSpecifier(op, evr_ver_dict)

Bases: object

A class to represent a version specifier with properties and methods to manipulate and retrieve version information.

op

The operation associated with the version specifier.

Type:

str

_evr_ver_dict

A dictionary containing epoch, version, and release information.

Type:

dict

property cpe_id

property ev_ver

static evr_dict_to_str(evr, fully_formed_evr_string=False)

Convert an EVR (Epoch-Version-Release) dictionary to a string representation.

Parameters:

• evr (dict) – A dictionary containing ‘epoch’, ‘version’, and ‘release’ keys. ‘epoch’ and ‘release’ can be None.

• fully_formed_evr_string (bool) – If True, ensures that the returned string includes ‘0’ for missing ‘epoch’ and ‘-0’ for missing ‘release’.

Returns:

The string representation of the EVR dictionary.

Return type:

str

property evr_op

property evr_ver

matches(evr: Dict)

Check if a given EVR dictionary satisfies this version specifier.

Parameters:

evr (dict) – A dictionary containing ‘epoch’, ‘version’, and ‘release’ keys.

Returns:

True if the EVR satisfies this version specifier, False otherwise.

Return type:

bool

property oval_id

property title

property ver

class ssg.utils.VersionSpecifierSet(s=())

Bases: set

A set-like collection that only accepts VersionSpecifier objects.

This class extends the built-in set and ensures that all elements are instances of the VersionSpecifier class. It provides additional properties to generate titles, CPE IDs, and OVAL IDs from the contained VersionSpecifier objects.

title

A string representation of the set, joining the titles of the contained VersionSpecifier objects with ‘ and ‘.

Type:

str

cpe_id

A string representation of the set, joining the CPE IDs of the contained VersionSpecifier objects with ‘:’.

Type:

str

oval_id

A string representation of the set, joining the OVAL IDs of the contained VersionSpecifier objects with ‘_’.

Type:

str

property cpe_id

property oval_id

property title

ssg.utils.apply_formatting_on_dict_values(source_dict, string_dict, ignored_keys=frozenset({}))

Apply string formatting on dictionary values.

This function uses Python’s built-in string replacement to replace strings marked by {token} if “token” is a key in the string_dict parameter. It skips keys in source_dict which are listed in the ignored_keys parameter. This function works only for dictionaries whose values are dictionaries or strings.

Parameters:

• source_dict (dict) – The source dictionary whose values need formatting.

• string_dict (dict) – A dictionary containing replacement strings for tokens.

• ignored_keys (frozenset, optional) – A set of keys to be ignored during formatting. Defaults to an empty frozenset.

Returns: dict: A new dictionary with formatted values.

ssg.utils.banner_anchor_wrap(banner_text)

Wraps the given banner text with ‘^’ at the beginning and ‘$’ at the end.

Parameters:

banner_text (str) – The text to be wrapped.

Returns:

The banner text wrapped with ‘^’ at the beginning and ‘$’ at the end.

Return type:

str

ssg.utils.banner_regexify(banner_text)

Converts a banner text into a regex pattern.

This function escapes special regex characters in the input text and then performs the following transformations: - Replaces newline characters (”

“) with a placeholder string “BFLMPSVZ”.

• Replaces spaces with a regex pattern that matches one or more whitespace characters or newline characters.

• Replaces the placeholder string “BFLMPSVZ” with a regex pattern that matches one or more newline characters or escaped newline sequences.

Args:

banner_text (str): The banner text to be converted into a regex pattern.

Returns:

str: The resulting regex pattern.

ssg.utils.check_conflict_regex_directory(data)

Validate that either all paths are directories or the ‘file_regex’ key exists.

This function checks the following conditions: 1. If the ‘is_directory’ key is present in the data dictionary, it ensures that all file paths

in the ‘filepath’ list are either directories or files. Mixing directories and files is not allowed.

2. If the ‘file_regex’ key is present in the data dictionary, it ensures that all file paths in the ‘filepath’ list are directories.

Parameters:

data (dict) – A dictionary containing the following keys: - ‘filepath’ (list): A list of file paths to be validated. - ‘is_directory’ (bool, optional): A flag indicating whether the paths are directories. - ‘file_regex’ (str, optional): A regular expression pattern for file matching. - ‘_rule_id’ (str): An identifier for the rule being validated.

Raises:

ValueError – If the file paths are a mix of directories and files, or if the ‘file_regex’ key is used but the file paths are not directories.

ssg.utils.comparison_to_oval(op)

Converts a comparison operator to its corresponding OVAL string representation.

Parameters:

op (str) – The comparison operator to convert. Expected values are ‘==’, ‘!=’, ‘>’, ‘<’, ‘>=’, ‘<=’.

Returns:

The OVAL string representation of the comparison operator.

Return type:

str

Raises:

KeyError – If the provided operator is not one of the expected values.

ssg.utils.ensure_file_paths_and_file_regexes_are_correctly_defined(data)

Ensures that the data structure for file paths and file regexes is correctly defined.

This function is used for the file_owner, file_groupowner, and file_permissions templates. It ensures that: - The ‘filepath’ item in the data is a list. - The number of items in ‘file_regex’ matches the number of items in ‘filepath’.

Note

• If ‘filepath’ is a string, it will be converted to a list containing that string.

• If ‘file_regex’ is a string, it will be converted to a list with the same length as ‘filepath’, with each element being the original ‘file_regex’ string.

• If there are multiple regexes for a single filepath, the filepath must be declared multiple times.

Parameters:

data (dict) – A dictionary containing file path and file regex information. It must contain the keys: - ‘filepath’: A string or list of strings representing file paths. - ‘file_regex’ (optional): A string or list of strings representing file regexes. - ‘_rule_id’: A string representing the rule ID.

Raises:

ValueError – If the number of items in ‘file_regex’ does not match the number of items in ‘filepath’.

ssg.utils.enum(*args)

Creates an enumeration with the given arguments.

Parameters:

*args – A variable length argument list of strings representing the names of the enumeration members.

Returns:

A new enumeration type with the given member names as attributes, each assigned a

unique integer value starting from 0.

Return type:

type

Example

>>> Colors = enum('RED', 'GREEN', 'BLUE')
>>> Colors.RED
0
>>> Colors.GREEN
1
>>> Colors.BLUE
2

ssg.utils.escape_comparison(op)

Maps a comparison operator to its corresponding string representation.

Parameters:

op (str) – The comparison operator to be mapped. Expected values are ‘==’, ‘!=’, ‘>’, ‘<’, ‘>=’, ‘<=’.

Returns:

The string representation of the comparison operator.

Possible return values are ‘eq’, ‘ne’, ‘gt’, ‘le’, ‘gt_or_eq’, ‘le_or_eq’.

Return type:

str

Raises:

KeyError – If the provided operator is not in the mapping dictionary.

ssg.utils.escape_id(text)

Converts a given text into a string that is more readable and compatible with OSCAP/XCCDF/OVAL entity IDs by replacing non-word characters with underscores and stripping leading/trailing underscores.

Parameters:

text (str) – The input string to be converted.

Returns:

The converted string with non-word characters replaced by underscores and

leading/trailing underscores removed.

Return type:

str

ssg.utils.escape_regex(text)

Escapes special characters in a given text to make it safe for use in regular expressions.

This function mimics the behavior of re.escape() in Python 3.7, which escapes a reasonable set of characters. Specifically, it escapes the following characters: #, $, &, *, +, ., ^, , |, ~, :, (, ), and -. Note that the characters ‘!’, ‘”’, ‘%’, “’”, ‘,’, ‘/’, ‘:’, ‘;’, ‘<’, ‘=’, ‘>’, ‘@’, and “” are not escaped.

Parameters:

text (str) – The input string containing characters to be escaped.

Returns:

A new string with special characters escaped.

Return type:

str

ssg.utils.escape_yaml_key(text)

Escapes uppercase letters and caret symbols in a YAML key by prefixing them with a caret and converts the entire key to lowercase.

This function is used to handle the limitation of OVAL’s name argument of the field type, which requires avoiding uppercase letters for keys. The probe would escape them with the ‘^’ symbol.

Example

myCamelCase^Key -> my^camel^case^^^key

Parameters:

text (str) – The YAML key to be escaped.

Returns:

The escaped and lowercased YAML key.

Return type:

str

ssg.utils.get_cpu_count()

Returns the most likely estimate of the number of CPUs in the machine for threading purposes, gracefully handling errors and possible exceptions.

Returns:

The number of CPUs available. If the number of CPUs cannot be determined, returns 2

as a default value.

Return type:

int

ssg.utils.get_fixed_product_version(product, product_version)

Adjusts the product version format for specific products.

Some product versions have a dot in between the numbers while the product ID doesn’t have the dot, but the full product name does. This function ensures the correct format for the product version.

Parameters:

• product (str) – The name of the product (e.g., “ubuntu”).

• product_version (str) – The version of the product as a string.

Returns:

The adjusted product version with the correct format.

Return type:

str

ssg.utils.is_applicable(platform, product)

Check if a platform is applicable for the given product.

This function determines whether a specified platform or a list of platforms is applicable for a given product. It handles cases where the platform is specified as ‘all’ or ‘multi_platform_all’, as well as cases where the platform is a comma-separated list of products.

Parameters:

• platform (str) – The platform or list of platforms to check.

• product (str) – The product to check applicability for.

Returns:

True if the product is applicable for the platform or list of platforms, False otherwise.

Return type:

bool

ssg.utils.is_applicable_for_product(platform, product)

Determines if a remediation script is applicable for a given product based on the platform specifier.

The function checks if the platform is either a general multi-platform specifier or matches the specific product name and version.

Parameters:

• platform (str) – The platform specifier of the remediation script.

• product (str) – The product name and version.

Returns:

True if the remediation script is applicable for the product, False otherwise.

Return type:

bool

ssg.utils.map_name(version)

Maps SSG Makefile internal product name to official product name.

This function takes a version string and maps it to an official product name based on predefined mappings. It handles multi-platform versions by trimming the “multi_platform_” prefix and recursively mapping the trimmed version.

Parameters:

version (str) – The version string to be mapped.

Returns:

The official product name corresponding to the given version.

Return type:

str

Raises:

RuntimeError – If the version is invalid or cannot be mapped to any known product.

ssg.utils.merge_dicts(left, right)

Merges two dictionaries, keeping left and right as passed.

If there are any common keys between left and right, the value from right is used.

Parameters:

• left (dict) – The first dictionary.

• right (dict) – The second dictionary.

Returns:

The merged dictionary containing keys and values from both left and right dictionaries.

Return type:

dict

ssg.utils.mkdir_p(path)

Create a directory and all intermediate-level directories if they do not exist.

Parameters:

path (str) – The directory path to create.

Returns:

True if the directory was created, False if it already exists.

Return type:

bool

Raises:

OSError – If the directory cannot be created and it does not already exist.

ssg.utils.name_to_platform(names)

Converts one or more full names to a string containing one or more <platform> elements.

Parameters:

names (str or list of str) – A single name as a string or a list of names.

Returns:

A string containing one or more <platform> elements.

Return type:

str

ssg.utils.parse_name(product)

Parses a given product string and returns a namedtuple containing the name and version.

Parameters:

product (str) – The product string to parse, e.g., “rhel9”.

Returns:

A namedtuple with ‘name’ and ‘version’ attributes, e.g., (“rhel”, “9”).

Return type:

namedtuple

Example

>>> parse_name("rhel9")
product(name='rhel', version='9')

ssg.utils.parse_platform(platform)

Parses a comma-separated string of platforms and returns a set of trimmed platform names.

Parameters:

platform (str) – A comma-separated string of platform names.

Returns:

A set of platform names with leading and trailing whitespace removed.

Return type:

set

ssg.utils.parse_template_boolean_value(data, parameter, default_value)

Parses a boolean value from a template parameter.

Parameters:

• data (dict) – The dictionary containing the template data.

• parameter (str) – The key for the parameter to parse.

• default_value (bool) – The default value to return if the parameter is not found or is None.

Returns:

The parsed boolean value.

Return type:

bool

Raises:

ValueError – If the parameter value is not “true” or “false”.

ssg.utils.product_to_name(prod)

Converts a product identifier into a full product name.

This function takes a product identifier and attempts to match it with a full product name from the FULL_NAME_TO_PRODUCT_MAPPING dictionary. If the product identifier is found in the dictionary, the corresponding full product name is returned. If the product identifier is in the MULTI_PLATFORM_LIST or is ‘all’, a string prefixed with “multi_platform_” is returned. If the product identifier is not recognized, a RuntimeError is raised.

Parameters:

prod (str) – The product identifier to convert.

Returns:

The full product name corresponding to the given product identifier.

Return type:

str

Raises:

RuntimeError – If the product identifier is not recognized.

ssg.utils.product_to_platform(prods)

Converts one or more product ids into a string with one or more <platform> elements.

Parameters:

prods (str or list of str) – A single product id as a string or a list of product ids.

Returns:

A string containing one or more <platform> elements.

Return type:

str

ssg.utils.read_file_list(path)

Reads the given file path and returns the contents as a list.

Parameters:

path (str) – The path to the file to be read.

Returns:

A list containing the contents of the file, split by lines or other delimiters as

defined by split_string_content.

Return type:

list

Raises:

• FileNotFoundError – If the file at the given path does not exist.

• IOError – If an I/O error occurs while reading the file.

ssg.utils.recurse_or_substitute_or_do_nothing(v, string_dict, ignored_keys=frozenset({}))

Recursively applies string formatting to dictionary values, substitutes strings, or returns the value unchanged.

Parameters:

• v (Any) – The value to process. Can be a dictionary, string, or any other type.

• string_dict (dict) – A dictionary containing string keys and values to be used for formatting.

• ignored_keys (frozenset, optional) – A set of keys to ignore when processing dictionaries. Defaults to an empty frozenset.

Returns:

The processed value. If v is a dictionary, returns a dictionary with formatted values.

If v is a string, returns the formatted string. Otherwise, returns v unchanged.

Return type:

Any

ssg.utils.required_key(_dict, _key)

Returns the value associated with the specified key in the given dictionary.

Parameters:

• _dict (dict) – The dictionary to search for the key.

• _key – The key to look for in the dictionary.

Returns:

The value associated with the specified key in the dictionary.

Raises:

ValueError – If the key is not found in the dictionary, an exception is raised with a message indicating that the key is required but was not found.

ssg.utils.select_templated_tests(test_dir_config, available_scenarios_basenames)

ssg.utils.sha256(text)

Generate a SHA-256 hash for the given text.

Parameters:

text (str) – The input text to be hashed.

Returns:

The SHA-256 hash of the input text as a hexadecimal string.

Return type:

str

ssg.utils.split_string_content(content)

Splits the input string content by newline characters and returns the result as a list of strings.

Parameters:

content (str) – The string content to be split.

Returns:

A list of strings, each representing a line from the input content.

The last element will be removed if it is an empty string.

Return type:

list

ssg.utils.subset_dict(dictionary, keys)

Restricts a dictionary to only include specified keys.

This function creates a new dictionary that contains only the key-value pairs from the original dictionary where the key is present in the provided list of keys. Neither the original dictionary nor the list of keys is modified.

Parameters:

• dictionary (dict) – The original dictionary from which to create the subset.

• keys (iterable) – An iterable of keys that should be included in the subset.

Returns:

A new dictionary containing only the key-value pairs where the key is in the

provided list of keys.

Return type:

dict

ssg.utils.write_list_file(path, contents)

Writes the given contents to the specified file path.

Parameters:

• path (str) – The file path where the contents will be written.

• contents (list of str) – A list of strings to be written to the file.

Returns:

None

ssg.variables module

ssg.variables.get_variable_files(content_dir: str) → List[str]

Retrieves all variable files from the specified content root directory.

This function iterates through a predefined list of benchmark directories and collects all variable files within those directories.

Parameters:

content_dir (str) – The root directory containing benchmark directories.

Returns:

A list of paths to all variable files found within the benchmark directories.

Return type:

list

ssg.variables.get_variable_files_in_folder(content_dir: str, subfolder: str) → List[str]

Retrieve a list of variable files within a specified folder in the project.

Parameters:

• content_dir (str) – The root directory of the content.

• subfolder (str) – The folder within the project to search for variable files.

Returns:

A list of paths to variable files found within the specified folder.

Return type:

list

ssg.variables.get_variable_options(content_dir: str, variable_id: str | None = None) → Dict[str, Dict[str, str]]

Retrieve the options for specific or all variables from the content root directory.

If variable_id is provided, returns options for that variable only. If variable_id is not provided, returns a dictionary of all variables with their options.

Parameters:

• content_dir (str) – The root directory containing benchmark directories.

• variable_id (str, optional) – The ID of the variable to retrieve options for. Defaults to None.

Returns:

If variable_id is None, a dictionary where keys are variable IDs and values are

their options. Otherwise, a dictionary of options for the specified variable.

Return type:

dict

ssg.variables.get_variable_property(content_dir: str, variable_id: str, property_name: str) → str

Retrieve a specific property of a variable from the content root directory.

Parameters:

• content_dir (str) – The root directory containing benchmark directories.

• variable_id (str) – The ID of the variable to retrieve the property for.

• property_name (str) – The name of the property to retrieve.

Returns:

The value of the specified property for the variable.

Return type:

str

ssg.variables.get_variable_values(content_dir: str, profiles_variables: dict) → dict

Update the variables dictionary with actual values for each variable option.

Given a content root directory and a dictionary of variables, this function retrieves the respective options values from variable files and updates the variables dictionary with these values.

Parameters:

• content_dir (str) – The root directory of the content.

• profiles_variables (dict) – A dictionary where keys are variable names and values are dictionaries of product-profile pairs.

Returns:

The updated variables dictionary with possible values for each variable.

Return type:

dict

ssg.variables.get_variables_by_products(content_dir: str, products: list) → Dict[str, dict]

Retrieve variables by products from the specified content root directory.

This function collects profiles for the given products and extracts variables from these profiles. If you already have a list of Profiles obtained by get_profiles_from_products() defined in profiles.py, consider to use get_variables_from_profiles() instead.

Parameters:

• content_dir (str) – The root directory of the content.

• products (list) – A list of products to retrieve variables for.

Returns:

A dictionary where keys are variable names and values are dictionaries of

product-profile pairs.

Return type:

dict

ssg.variables.get_variables_from_profiles(profiles: list) → Dict[str, Dict[str, str]]

Extracts variables from a list of profiles and organizes them into a nested dictionary.

Parameters:

profiles (list) – A list of profile objects, each containing selections and id attributes.

Returns:

A nested dictionary where the first level keys are variable names, the second level

keys are product names, and the third level keys are profile IDs, with the corresponding values being the variable values.

Return type:

dict

ssg.xccdf module

A couple generic XCCDF utilities used by build_all_guides.py and build_all_remediations.py

Author: Martin Preisler <mpreisle@redhat.com>

ssg.xccdf.get_benchmark_id_title_map(input_tree)

Extracts a mapping of benchmark IDs to their titles from an XML tree.

Parameters:

input_tree (xml.etree.ElementTree.ElementTree) – The XML tree containing benchmark data.

Returns:

A dictionary where the keys are benchmark IDs (str) and the values are benchmark

titles (str).

Return type:

dict

ssg.xccdf.get_profile_choices_for_input(input_tree, benchmark_id, tailoring_tree)

Returns a dictionary that maps profile_ids to their respective titles.

Parameters:

• input_tree (ElementTree) – The XML tree containing the benchmark profiles.

• benchmark_id (str) – The ID of the benchmark to filter profiles.

• tailoring_tree (ElementTree, optional) – An optional XML tree containing tailored profiles.

Returns:

A dictionary where keys are profile IDs and values are profile titles.

Return type:

dict

ssg.xccdf.get_profile_short_id(long_id)

Shortens the given profile ID if it matches the XCCDF 1.2 long ID format.

Parameters:

long_id (str) – The long profile ID to be shortened.

Returns:

The shortened profile ID if the long ID matches the XCCDF 1.2 format, otherwise

returns the original long ID.

Return type:

str

ssg.xccdf.scrape_benchmarks(root, namespace, dest)

Add all benchmark elements in root to dest list.

This function searches for all elements with the tag ‘Benchmark’ within the given XML root element, using the specified namespace. It then adds these elements to the destination list ‘dest’ along with their namespace. If the root element itself is a ‘Benchmark’, it is also added to the list.

Parameters:

• root (xml.etree.ElementTree.Element) – The root XML element to search within.

• namespace (str) – The XML namespace to use when searching for ‘Benchmark’ elements.

• dest (list) – The list to which found ‘Benchmark’ elements and their namespace will be added.

Returns:

None

ssg.xml module

Common functions for processing XML in SSG

class ssg.xml.XMLBenchmark(root)

Bases: XMLElement

Represents an XCCDF Benchmark read from an XML file.

root

The root element of the XML document.

Type:

Element

find_all_cpe_platforms(idref)

Find all CPE platforms with the given idref.

Parameters:

idref (str) – The reference ID of the CPE platform to find.

Returns:

A list of XMLCPEPlatform objects that match the given idref.

Return type:

list

find_rule(rule_id)

Find and return an XMLRule object for the given rule ID.

Parameters:

rule_id (str) – The ID of the rule to find.

Returns:

An XMLRule object if the rule is found, otherwise None.

Return type:

XMLRule

find_rules(rule_id)

Find and return rules based on the given rule_id.

Parameters:

rule_id (str) – The ID of the rule to find. If None, all rules are returned.

Returns:

A list of XMLRule objects that match the given rule_id.

If rule_id is None, returns all rules.

Return type:

list

Raises:

ValueError – If no rules are found for the given rule_id.

class ssg.xml.XMLCPEPlatform(root)

Bases: XMLElement

A class to represent an XML CPE Platform element.

root

The root element of the XML tree.

Type:

xml.etree.ElementTree.Element

find_all_check_fact_ref_elements()

class ssg.xml.XMLComponent(root)

Bases: XMLElement

Represents the element of the Datastream component that has relevant content.

This makes it easier to access contents pertaining to a SCAP component.

find_boolean_question(ocil_id)

Finds and returns the text of a boolean question from an OCIL questionnaire.

Parameters:

ocil_id (str) – The ID of the OCIL questionnaire.

Returns:

The text of the boolean question.

Return type:

str

Raises:

ValueError – If the OCIL questionnaire, test action, or boolean question does not exist.

find_ocil_boolean_question(question_id)

Find an OCIL boolean question by its ID.

Parameters:

question_id (str) – The ID of the boolean question to find.

Returns:

An instance of XMLOcilQuestion representing the found boolean question.

Return type:

XMLOcilQuestion

Raises:

AttributeError – If the question is not found or if the XML structure is incorrect.

find_ocil_questionnaire(def_id)

Finds and returns an OCIL questionnaire by its definition ID.

Parameters:

def_id (str) – The definition ID of the OCIL questionnaire to find.

Returns:

An instance of XMLOcilQuestionnaire representing the found

questionnaire.

Return type:

XMLOcilQuestionnaire

Raises:

AttributeError – If the ‘ocil:questionnaires’ or ‘ocil:questionnaire’ elements are not found.

find_ocil_test_action(test_action_ref)

Finds and returns an OCIL test action based on the provided reference ID.

Parameters:

test_action_ref (str) – The reference ID of the test action to find.

Returns:

An instance of XMLOcilTestAction representing the found test action.

Return type:

XMLOcilTestAction

Raises:

AttributeError – If the test action is not found in the XML structure.

find_oval_definition(def_id)

Find and return an OVAL definition by its ID.

Parameters:

def_id (str) – The ID of the OVAL definition to find.

Returns:

An instance of XMLOvalDefinition representing the found definition.

Return type:

XMLOvalDefinition

Raises:

AttributeError – If the definitions or definition element is not found.

class ssg.xml.XMLContent(root)

Bases: XMLElement

XMLContent is a class that represents a Data Stream or an XCCDF Benchmark read from an XML file.

check_engines

A list of tuples containing check engine names and their corresponding XML tags.

Type:

list

check_engines = [('OVAL', 'oval:oval_definitions'), ('OCIL', 'ocil:ocil')]

find_benchmark(id_)

Finds and returns an XMLBenchmark object for the given benchmark ID.

This method searches for a benchmark with the specified ID within the XML structure. It first looks for “ds:component” elements and checks if any of them contain a “Benchmark” element with the given ID. If no “ds:component” elements are found, it checks if the root element is a benchmark itself.

Parameters:

id (str) – The ID of the benchmark to find.

Returns:

An XMLBenchmark object if a benchmark with the given ID is found,

otherwise None.

Return type:

XMLBenchmark

get_benchmarks()

Extracts and yields XMLBenchmark objects from the XML tree.

This method searches for ‘ds:component’ elements in the XML tree. If no such elements are found, it checks if the root element is a benchmark and yields an XMLBenchmark object if true. Otherwise, it iterates over each ‘ds:component’ element and searches for ‘Benchmark’ elements within the component, yielding an XMLBenchmark object for each found benchmark.

Yields:

XMLBenchmark – An instance of XMLBenchmark for each found benchmark in the XML tree.

get_component_refs()

Extracts and returns a dictionary of component references from the XML data.

This method searches for all “ds:component-ref” elements within “ds:checks” elements in the XML data stream. It retrieves the “href” attribute from the “xlink” namespace and the “id” attribute from each “ds:component-ref” element and stores them in a dictionary.

Returns:

A dictionary where the keys are the “href” attributes and the values are the

”id” attributes of the “ds:component-ref” elements.

Return type:

dict

get_uris()

Extracts URIs and their corresponding names from the XML data.

This method searches through the XML structure defined in self.root for data streams, checklists, and catalogs to find URI elements. It then extracts the ‘uri’ and ‘name’ attributes from each URI element and stores them in a dictionary.

Returns:

A dictionary where the keys are URI strings and the values are the corresponding

names.

Return type:

dict

is_benchmark()

Determines if the root tag of the XML document is a Benchmark tag.

This method checks if the root tag of the XML document matches the Benchmark tag for either the “xccdf-1.2” or “xccdf-1.1” namespace. If the root tag matches the “xccdf-1.2” namespace, it returns True. If the root tag matches the “xccdf-1.1” namespace, it sets the content_xccdf_ns attribute to “xccdf-1.1” and returns True.

Returns:

True if the root tag is a Benchmark tag for either “xccdf-1.2” or “xccdf-1.1”

namespace, False otherwise.

Return type:

bool

class ssg.xml.XMLElement(root)

Bases: object

Represents a generic element read from an XML file.

ns

A dictionary mapping namespace prefixes to their respective URIs.

Type:

dict

root

The root element of the XML structure.

Type:

Element

content_xccdf_ns

The XCCDF version namespace determined from the XML.

Type:

str

get_attr(attr)

Retrieve the value of an attribute from the root element.

Parameters:

attr (str) – The name of the attribute to retrieve.

Returns:

The value of the attribute if it exists, otherwise None.

Return type:

str or None

get_namespace()

Extracts and returns the XML namespace from the root tag of the XML document.

Returns:

The namespace URI extracted from the root tag.

Return type:

str

Raises:

AttributeError – If the root tag does not contain a namespace.

ns = {'catalog': 'urn:oasis:names:tc:entity:xmlns:xml:catalog', 'cpe-lang': 'http://cpe.mitre.org/language/2.0', 'ds': 'http://scap.nist.gov/schema/scap/source/1.2', 'ocil': 'http://scap.nist.gov/schema/ocil/2.0', 'oval': 'http://oval.mitre.org/XMLSchema/oval-definitions-5', 'xccdf-1.1': 'http://checklists.nist.gov/xccdf/1.1', 'xccdf-1.2': 'http://checklists.nist.gov/xccdf/1.2', 'xlink': 'http://www.w3.org/1999/xlink'}

class ssg.xml.XMLOcilQuestion(root)

Bases: XMLComponent

A class to represent an OCIL question in XML format.

root

The root element of the XML structure.

Type:

Element

ns

The namespace dictionary for XML parsing.

Type:

dict

get_question_test_element()

Retrieves the ‘question_text’ element from the XML tree.

This method searches for the ‘question_text’ element within the XML tree using the specified namespace.

Returns:

The ‘question_text’ element if found, otherwise None.

Return type:

Element

class ssg.xml.XMLOcilQuestionnaire(root)

Bases: XMLComponent

A class to represent an OCIL Questionnaire in XML format.

root

The root element of the XML structure.

Type:

Element

ns

A dictionary of XML namespaces.

Type:

dict

get_test_action_ref_element()

Retrieves the test action reference element from the XML.

This method searches for the ‘ocil:test_action_ref’ element within the ‘ocil:actions’ section of the XML document using the specified namespace.

Returns:

The found ‘ocil:test_action_ref’ element, or None if not found.

Return type:

Element

class ssg.xml.XMLOcilTestAction(root)

Bases: XMLComponent

A class to represent an OCIL Test Action in XML format.

root

The root element of the XML structure.

Type:

Element

class ssg.xml.XMLOvalDefinition(root)

Bases: XMLComponent

A class to represent an OVAL definition in XML format.

root

The root element of the XML document.

Type:

xml.etree.ElementTree.Element

ns

A dictionary of XML namespaces.

Type:

dict

get_criteria_element()

Retrieves the first ‘oval:criteria’ element from the XML document.

Returns:

The first ‘oval:criteria’ element found in the XML document, or None if no such element is found.

Return type:

xml.etree.ElementTree.Element

get_elements()

Extracts and returns a list of elements from the criteria element.

The method iterates over the children of the criteria element and identifies the tag of each child. Depending on the tag, it extracts relevant attributes and appends them to the elements list as tuples.

Returns:

A list of tuples where each tuple contains the element type and its associated

attribute value. The possible element types and their attributes are: - (“criteria”, operator) - (“criterion”, test_id) - (“extend_definition”, extend_def_id)

Return type:

list

class ssg.xml.XMLRule(root)

Bases: XMLElement

Represents an XCCDF Rule read from an XML file.

root

The root element of the XML tree.

Type:

Element

content_xccdf_ns

The namespace for XCCDF content.

Type:

str

ns

The namespace dictionary for XML parsing.

Type:

dict

get_all_platform_elements()

Retrieve all platform elements from the XML document.

This method searches for all elements with the tag ‘platform’ within the XML document’s root, using the specified namespace.

Returns:

A list of all found platform elements.

Return type:

list

get_check_content_ref_element(check_element)

Retrieves the ‘check-content-ref’ element from the given check element.

Parameters:

check_element (Element) – The XML element representing the check.

Returns:

The ‘check-content-ref’ sub-element if found, otherwise None.

Return type:

Element

get_check_element(check_system_uri)

Retrieve a check element from the XML tree based on the given check system URI.

Parameters:

check_system_uri (str) – The URI of the check system to find.

Returns:

The XML element corresponding to the check system URI, or None if not found.

Return type:

Element

get_element_text(el)

Extracts and returns the text content of an XML element.

If the element’s tag (without namespace) is “description”, it uses a specialized method to get the description text. Otherwise, it concatenates all text within the element.

Parameters:

el (xml.etree.ElementTree.Element) – The XML element from which to extract text.

Returns:

The text content of the XML element.

Return type:

str

get_fix_element(fix_uri)

Retrieve the ‘fix’ element from the XML tree based on the provided fix URI.

Parameters:

fix_uri (str) – The URI of the fix to be retrieved.

Returns:

The XML element corresponding to the fix URI, or None if not found.

Return type:

Element

get_version_element()

Retrieve the version element from the XML document.

This method searches for the version element within the XML document using the specified namespace.

Returns:

The version element if found, otherwise None.

Return type:

Element

join_text_elements()

Collects and concatenates text from relevant subelements of the root element.

This function iterates over the subelements of the root element, collects their text, and concatenates it into a single string. It skips certain elements that are not relevant for comparison, such as “fix” elements and “reference” elements with specific attributes. For each collected text, it injects a line indicating the tag of the element from which the text was collected to facilitate tracking.

Returns:

A concatenated string of text from relevant subelements, with injected lines

indicating the source element tags.

Return type:

str

ssg.xml.add_xhtml_namespace(data)

Given an XML blob, adds the XHTML namespace to all relevant tags.

This function performs two main transformations: 1. It transforms <tt> tags into <code> tags. 2. It adds the XHTML prefix to specified elements.

Parameters:

data (str) – The XML data as a string.

Returns:

The modified XML data with XHTML namespaces added.

Return type:

str

ssg.xml.determine_xccdf_tree_namespace(tree)

Determines the XCCDF namespace of the given XML tree.

Parameters:

tree (xml.etree.ElementTree.ElementTree) – The XML tree to examine.

Returns:

The XCCDF namespace URI.

Return type:

str

Raises:

ValueError – If the root element of the tree is not recognized as a Benchmark element from either XCCDF 1.1 or XCCDF 1.2 namespaces.

ssg.xml.get_element_namespace(self)

Extracts the namespace from the root element’s tag.

The method uses a regular expression to search for a namespace pattern in the root element’s tag. The namespace is expected to be enclosed in curly braces at the beginning of the tag.

Returns:

The namespace extracted from the root element’s tag.

Return type:

str

Raises:

AttributeError – If the root element’s tag does not match the expected pattern.

ssg.xml.get_element_tag_without_ns(xml_tag)

Extracts the tag name from an XML element, removing any namespace.

Parameters:

xml_tag (str) – The XML tag with namespace.

Returns:

The XML tag without the namespace.

Return type:

str

Raises:

AttributeError – If the input string does not match the expected pattern.

ssg.xml.get_namespaces_from(file)

Extracts and returns a dictionary of XML namespaces from the given file.

Parameters:

file (str or file-like object) – The path to the XML file or a file-like object containing XML data.

Returns:

A dictionary where the keys are namespace prefixes and the values are namespace URIs.

Returns an empty dictionary if an error occurs during parsing.

Return type:

dict

Return dictionary of namespaces in file. Return empty dictionary in case of error.

ssg.xml.map_elements_to_their_ids(tree, xpath_expr)

Given an ElementTree and an XPath expression, iterate through matching elements and create 1:1 id->element mapping.

Parameters:

• tree (ElementTree) – The XML tree to search within.

• xpath_expr (str) – The XPath expression to match elements.

Raises:

AssertionError – If a matching element doesn’t have the id attribute.

Returns:

A dictionary mapping element IDs to their corresponding elements.

Return type:

dict

ssg.xml.open_xml(filename)

Open and parse an XML file.

This function registers all possible namespaces and then parses the XML file specified by the given filename, returning the resulting XML tree.

Parameters:

filename (str) – The path to the XML file to be parsed.

Returns:

The parsed XML tree.

Return type:

xml.etree.ElementTree.ElementTree

Raises:

xml.etree.ElementTree.ParseError – If there is an error parsing the XML file.

ssg.xml.oval_generated_header(product_name, schema_version, ssg_version)

Generates an OVAL header for a given product.

Parameters:

• product_name (str) – The name of the product.

• schema_version (str) – The version of the OVAL schema.

• ssg_version (str) – The version of the SCAP Security Guide (SSG).

Returns:

A string containing the OVAL header with the provided product name, schema version,

SSG version, Python version, and the current timestamp.

Return type:

str

ssg.xml.parse_file(filename)

Parses an XML file and returns the root element of the ElementTree.

Parameters:

filename (str) – The path to the XML file to be parsed.

Returns:

The root element of the parsed XML tree.

Return type:

xml.etree.ElementTree.Element

ssg.xml.register_namespaces(ns=None)

Register all possible namespaces.

This function registers XML namespaces for use with the ElementTree module. If no namespaces are provided, it defaults to using the PREFIX_TO_NS dictionary.

Parameters:

ns (dict, optional) – A dictionary mapping prefixes to namespace URIs. If None, the function uses the PREFIX_TO_NS dictionary.

Raises:

Exception – Catches all exceptions, which may occur if using an old version of Python. This is non-essential and will be silently ignored.

ssg.yaml module

Common functions for processing YAML in SSG

exception ssg.yaml.DocumentationNotComplete

Bases: Exception

ssg.yaml.convert_string_to_bool(string)

Converts a string representation of a boolean to its corresponding boolean value.

Parameters:

string (str) – The string to convert. Expected values are “true” or “false” (case insensitive).

Returns:

True if the string is “true” (case insensitive), False if the string is “false”

(case insensitive).

Return type:

bool

Raises:

ValueError – If the string is not “true” or “false” (case insensitive).

ssg.yaml.open_and_expand(yaml_file, substitutions_dict=None)

Process the given YAML file as a template, using the provided substitutions dictionary to perform variable expansion. After expanding the template, process the result as YAML content.

Parameters:

• yaml_file (str) – The path to the YAML file to be processed.

• substitutions_dict (dict, optional) – A dictionary containing key-value pairs for template substitution. Defaults to an empty dictionary if not provided.

Returns:

The processed YAML content as a dictionary.

Return type:

dict

Raises:

yaml.scanner.ScannerError – If there is an error in scanning the YAML content, typically due to incorrect indentation after template expansion.

See also

_open_yaml: Function to open and parse the YAML content.

ssg.yaml.open_and_macro_expand_from_dir(yaml_file, content_dir, substitutions_dict=None)

Opens a YAML file and expands macros from a specified directory. It is similar to open_and_expand but loads macro definitions from a specified directory instead of the default directory defined in constants. This is useful in cases where the SSG library is consumed by an external project.

Parameters:

• yaml_file (str) – The path to the YAML file to be opened and expanded.

• content_dir (str) – The content dir directory to be used for expansion.

• substitutions_dict (dict, optional) – A dictionary of substitutions to be used for macro expansion. If None, a new dictionary will be created from the content_dir.

Returns:

The expanded content of the YAML file.

Return type:

dict

ssg.yaml.open_raw(yaml_file)

Open the given YAML file and parse its contents without performing any template processing.

Parameters:

yaml_file (str) – The path to the YAML file to be opened and parsed.

Returns:

The parsed contents of the YAML file.

Return type:

dict

See also

_open_yaml: The function used to parse the YAML contents.

ssg.yaml.ordered_dump(data, stream=None, Dumper=<class 'yaml.cyaml.CDumper'>, **kwds)

Serializes a Python object into a YAML stream, preserving the order of dictionaries.

This function is a drop-in replacement for yaml.dump(), but it ensures that the order of dictionaries is preserved. It also includes custom representers for dictionaries and strings, and applies specific formatting fixes for YAML output.

Parameters:

• data (Any) – The Python object to serialize.

• stream (Optional[IO]) – The stream to write the YAML output to. If None, the YAML string is returned.

• Dumper (yaml.Dumper) – The YAML dumper class to use. Defaults to yaml_Dumper.

• **kwds – Additional keyword arguments to pass to yaml.dump().

Returns:

The YAML string if stream is None, otherwise None.

Return type:

Optional[str]

ssg.yaml.ordered_load(stream, Loader=<class 'yaml.cyaml.CLoader'>, object_pairs_hook=<class 'collections.OrderedDict'>)

Load a YAML stream while preserving the order of dictionaries.

This function is a drop-in replacement for yaml.load(), but it ensures that the order of dictionaries is preserved by using OrderedDict.

Parameters:

• stream (str) – The YAML stream to load.

• Loader (yaml.Loader, optional) – The YAML loader class to use. Defaults to yaml_Loader.

• object_pairs_hook (callable, optional) – A callable that will be used to construct ordered mappings. Defaults to OrderedDict.

Returns:

The loaded YAML data with preserved order of dictionaries.

Return type:

OrderedDict

Raises:

yaml.scanner.ScannerError – If there is an error when trying to load the stream.

Notes

• If there is a ScannerError, additional hints will be printed to help diagnose common issues such as unquoted colons or other special symbols in the stream.

• The function will exit the program with status code 1 if a ScannerError occurs.

ssg.yaml.update_yaml_list_or_string(current_contents, new_contents, prepend=False)

Update a YAML list or string by combining current and new contents.

This function takes the current contents and new contents, both of which can be either a string or a list of strings, and combines them into a single list or string. If the prepend flag is set to True, the new contents are added before the current contents. Otherwise, the new contents are appended to the current contents.

Parameters:

• current_contents (str or list of str) – The existing contents to be updated.

• new_contents (str or list of str) – The new contents to be added.

• prepend (bool) – If True, new contents are added before current contents. Defaults to False.

Returns:

The updated contents. If the result is a single item, it is returned

as a string. If the result is empty, an empty string is returned.

Return type:

str or list of str

Module contents