Gemara + complyctl: Full Walkthrough with File Inspection

End-to-end manual walkthrough for NIST 800-53 compliance scanning on RHEL9. Shows every file produced at each step with real content excerpts.

---

Repository files involved

utils/nist_sync/
  export_to_gemara.py              # Step 2 — generates Gemara YAML from CaC content
  generate_complyctl_bundle.py     # Step 3 — builds per-baseline OCI bundle + pushes
  download_oscal.py                # Optional — enriches guidance_catalog with NIST prose

  vagrant/
    Vagrantfile                    # RHEL9 scanner VM definition (libvirt)
    populate_inventory.sh          # Extracts VM IP → writes ansible/inventory.ini

  ansible/
    inventory.ini                  # AUTO-GENERATED — host address + SSH key
    setup.yml                      # One-time VM setup (complyctl, provider, registry)
    scan.yml                       # Orchestrates Steps 2–7 across all baselines
    tasks/scan_baseline.yml        # Per-baseline subtasks called by scan.yml
    templates/complytime.yaml.j2   # complyctl config template written to VM

docs/manual/developer/
  15_nist_800_53_gemara_walkthrough.md  # This file
  16_nist_800_53_gemara_testing.md      # Full prerequisite + usage guide

products/rhel9/
  profiles/nist_800_53.profile     # AUTO-GENERATED by export_to_gemara.py — gitignored, do not edit

controls/nist_800_53.yml           # NIST 800-53 control → rule mappings (source of truth)

build/                             # Generated — gitignored
  ssg-rhel9-ds.xml                 # Step 1 output — SCAP data stream with nist_800_53 profile
  gemara/
    guidance_catalog.yaml          # Step 2 output — NIST control prose (needs OSCAL data)
    rhel9/
      control_catalog.yaml         # Step 2 output — controls + rule IDs + applicability
      rules_mapping.yaml           # Step 2 output — rule IDs ↔ NIST control IDs
  gemara-bundle/rhel9/{baseline}/
    rhel9_policy.yaml              # Step 3 output — complyctl Policy (assessment-plans)
    rhel9_catalog.yaml             # Step 3 output — filtered ControlCatalog for baseline
    complytime.yaml                # Step 3 reference — local test config (not used by Ansible)
    HOWTO.txt                      # Step 3 output — usage instructions
  complyctl-results/rhel9/{baseline}/
    arf.xml                        # Step 7 output — OpenSCAP Assessment Results Format
    results.xml                    # Step 7 output — XCCDF benchmark with rule results
    evaluation-log-*.yaml          # Step 7 output — complyctl structured evaluation log
    report.html                    # Step 8 output — oscap HTML report (human-readable)

---

Step 0 — Prerequisites

Tools needed (on your host machine)

Tool	Purpose	Get it
Python 3 + ruamel.yaml	Run export/bundle scripts	pip install ruamel.yaml
complyctl 1.0.0	Fetch/generate/scan workflow	~/bin/complyctl
complyctl-provider-openscap	OpenSCAP backend for complyctl	~/.local/share/complytime/providers/
oras	Pushes OCI bundles to registry	dnf install oras
Vagrant + vagrant-libvirt	RHEL9 VM (Vagrant workflow)	vagrant plugin install vagrant-libvirt
Ansible ≥ 2.14	Orchestrates VM setup + scan	pip install ansible

# Verify all tools before starting
complyctl version           # should print 1.0.0
oras version                # should print oras/v1.x
vagrant --version
ansible --version
python3 -c "import ruamel.yaml; print('ok')"

---

Step 1 — Build the SCAP data stream

The data stream is the source of XCCDF rule definitions. The system RPM (scap-security-guide) does NOT contain the nist_800_53 profile — always build from source.

Prerequisite — generate the profile first (Step 2 does this automatically):

products/rhel9/profiles/nist_800_53.profile is not committed to the repository. It is generated by export_to_gemara.py (Step 2) and listed in .gitignore. The generated content is deterministic and trivial — it just selects every rule touched by controls/nist_800_53.yml via the nist_800_53:all selector. complyctl then narrows the selection to one baseline using the Gemara Policy’s assessment-plans.

Run Step 2 first (or standalone):

source ./.pyenv.sh
python3 utils/nist_sync/export_to_gemara.py --products rhel9 --output-dir build/gemara

This writes (among other files):

# products/rhel9/profiles/nist_800_53.profile  (gitignored — do not commit)
documentation_complete: true
title: 'NIST SP 800-53 Rev 5'
description: |-
    Contains all rules mapped to NIST SP 800-53 Revision 5 controls in
    ComplianceAsCode for Red Hat Enterprise Linux 9, across all baselines
    (Low, Moderate, High).

    Generated by utils/nist_sync/export_to_gemara.py. Do not edit manually.
platform: rhel9
selections:
    - nist_800_53:all

Command:

./build_product rhel9 --datastream

Output:

build/ssg-rhel9-ds.xml     (~28 MB)

Verify the profile is present:

grep -c 'nist_800_53' build/ssg-rhel9-ds.xml
# should print a non-zero number

---

Step 2 — Generate Gemara artifacts

Reads the NIST 800-53 control mappings from the CaC content and produces three Gemara-schema YAML files.

Source files read:

• controls/nist_800_53.yml — control IDs, levels (low/moderate/high), and rule lists

• products/rhel9/product.yml — product metadata

• utils/nist_sync/data/nist_oscal.json — NIST prose (optional; download with download_oscal.py)

Command:

source ./.pyenv.sh
python3 utils/nist_sync/export_to_gemara.py \
    --products rhel9 \
    --output-dir build/gemara \
    --data-dir utils/nist_sync/data \
    --validate

Output — three files:

build/gemara/rhel9/control_catalog.yaml (ControlCatalog)

Maps each NIST control to its CaC rule IDs and baseline applicability. One controls: entry per control ID. Each entry lists assessment-requirements (the rule checks that satisfy the control), with applicability showing which baselines require it (low, moderate, or high).

metadata:
  id: nist-800-53-rev5-rhel9
  type: ControlCatalog
  gemara-version: 1.2.0
  description: NIST Special Publication 800-53 Revision 5 controls for RHEL9, generated from ComplianceAsCode
  applicability-groups:
  - id: rhel9-low                       # ← product-scoped: "rhel9-" prefix avoids collisions
    title: RHEL9 Low Baseline           #   when catalogs from multiple products coexist
    description: NIST 800-53 Low impact baseline for RHEL9
  - id: rhel9-moderate
    title: RHEL9 Moderate Baseline
    description: NIST 800-53 Moderate impact baseline for RHEL9 (inherits Low)
  - id: rhel9-high
    title: RHEL9 High Baseline
    description: NIST 800-53 High impact baseline for RHEL9 (inherits Low, Moderate)

title: NIST Special Publication 800-53 Revision 5 for RHEL9
groups:
- id: ac
  title: Access Control
  # ... 20 control families ...

controls:
- id: ac-2.5
  group: ac
  title: Inactivity Logout
  objective: 'Require that users log out when {{ insert: param, ac-02.05_odp }}.'
  state: Active
  assessment-requirements:
  - id: accounts_tmout                  # ← bare rule name (identity of the rule itself)
    state: Active
    text: "Rule 'accounts_tmout' MUST be verified"
    applicability: [rhel9-moderate]     # ← only required from moderate baseline up
  - id: no_invalid_shell_accounts_unlocked
    state: Active
    text: "Rule 'no_invalid_shell_accounts_unlocked' MUST be verified"
    applicability: [rhel9-moderate]
  - id: no_password_auth_for_systemaccounts
    state: Active
    text: "Rule 'no_password_auth_for_systemaccounts' MUST be verified"
    applicability: [rhel9-moderate]
  # ...

ID design — ControlCatalog vs MappingDocument:

assessment-requirements[].id uses the bare rule name (accounts_tmout) because it identifies the rule itself within a control. The rules_mapping.yaml uses compound IDs (ac-2.5--accounts_tmout) because a mapping entry identifies the relationship between a control and a rule — the same rule can appear under multiple controls and each (control, rule) pair is a distinct relationship.

build/gemara/rhel9/rules_mapping.yaml (MappingDocument)

Bidirectional index: given a CaC rule ID, find which NIST controls it satisfies. Used after scanning to trace a rule PASS/FAIL back to specific controls.

metadata:
  id: nist-800-53-rev5-rhel9-rules-mapping
  type: MappingDocument
  gemara-version: 1.2.0

source-reference:
  entry-type: Control          # ← "source" = the NIST control
target-reference:
  entry-type: AssessmentRequirement  # ← "target" = the CaC rule

mappings:
- id: ac-2.5--accounts_tmout
  source: ac-2.5               # NIST control ID
  relationship: implements
  targets:
  - entry-id: accounts_tmout   # CaC rule (short name, no prefix)
    strength: 8
    confidence-level: High
    rationale: Automated enforcement via ComplianceAsCode rule

- id: ac-2.5--no_invalid_shell_accounts_unlocked
  source: ac-2.5
  relationship: implements
  targets:
  - entry-id: no_invalid_shell_accounts_unlocked
    strength: 8
    confidence-level: High
    rationale: Automated enforcement via ComplianceAsCode rule
  # ... (hundreds more mappings) ...

build/gemara/guidance_catalog.yaml (GuidanceCatalog)

Optional — only generated when OSCAL data is present (download_oscal.py). Contains the official NIST prose for each control (the “what should be” layer).

metadata:
  id: nist-800-53-rev5-guidance
  type: GuidanceCatalog
  gemara-version: 1.2.0
  author:
    id: nist
    name: National Institute of Standards and Technology

controls:
- id: ac-2.5
  title: Inactivity Logout
  objective: >
    Require that users log out when [Assignment: organization-defined time period
    of expected inactivity or description of when to log out].
  guidance: >
    Inactivity logout is behavior- or policy-based and requires users to take
    physical action to log out when they are expecting inactivity longer than
    the defined period.
  # ...

Verify the export:

python3 utils/nist_sync/test_gemara_export.py --products rhel9
# prints: PASS for all three Gemara document types

---

Step 3 — Generate per-baseline OCI bundle

For each baseline (low / moderate / high), generate a filtered Policy that contains only the rules applicable to that baseline, then push to an OCI registry.

Command (example: moderate baseline):

python3 utils/nist_sync/generate_complyctl_bundle.py \
    --product rhel9 \
    --gemara-dir build/gemara \
    --output-dir build/gemara-bundle/rhel9/moderate \
    --baseline moderate \
    --base-profile nist_800_53 \
    --registry 127.0.0.1:5500 \
    --tag nist-800-53-rev5-rhel9-moderate:latest \
    --push --verbose

Output — four files written, one bundle pushed:

build/gemara-bundle/rhel9/moderate/rhel9_policy.yaml (Policy)

The file complyctl reads to know which rules to evaluate. Each assessment-plans entry maps to one OpenSCAP rule check. The id field must be the short CaC rule name (no xccdf_org.ssgproject.content_rule_ prefix).

title: NIST SP 800-53 Rev 5 for Red Hat Enterprise Linux 9
metadata:
  id: nist-800-53-rev5-rhel9-policy
  type: Policy
  gemara-version: 1.2.0
  description: >
    Automated evaluation policy for NIST SP 800-53 Rev 5 on RHEL9.
    requirement-id values are short CaC rule names (the OpenSCAP provider adds
    the xccdf_org.ssgproject.content_rule_ prefix).

imports:
  catalogs:
  - reference-id: nist-800-53-rev5-rhel9

adherence:
  evaluation-methods:
  - id: openscap-automated
    type: Behavioral
    mode: Automated
    executor:
      id: openscap
      name: OpenSCAP

  assessment-plans:
  - id: accounts_tmout               # ← short CaC rule name
    requirement-id: accounts_tmout   # ← same value (required by go-gemara)
    frequency: on-demand
    evaluation-methods:
    - id: openscap-automated
      type: Behavioral
      mode: Automated

  - id: configure_custom_crypto_policy_cis
    requirement-id: configure_custom_crypto_policy_cis
    # ...

  # 22 total assessment-plans for moderate baseline

Why 22 rules for moderate? The generator reads applicability from control_catalog.yaml and includes only rules where applicability contains the product-scoped baseline key (e.g., rhel9-moderate). Because baselines inherit upward, rules in the rhel9-low applicability group are already covered by a lower-baseline bundle — the moderate bundle only adds the rules that first appear at moderate level.

• rhel9-low baseline → 383 rules (rules applicable to low-impact systems)

• rhel9-moderate baseline → 22 rules (rules first required at moderate level)

• rhel9-high baseline → 4 rules (rules first required at high level)

build/gemara-bundle/rhel9/moderate/rhel9_catalog.yaml (ControlCatalog)

A subset of control_catalog.yaml filtered to the moderate baseline’s controls. Bundled alongside the Policy so complyctl has the full control context.

metadata:
  id: nist-800-53-rev5-rhel9
  type: ControlCatalog
  gemara-version: 1.2.0
  # ... same header as the full control_catalog.yaml ...

controls:
# Only controls that have assessment-requirements with applicability: [moderate]
- id: ac-2.5
  group: ac
  title: Inactivity Logout
  # ...

OCI bundle pushed to registry

The bundle is pushed as a two-layer OCI artifact:

nist-800-53-rev5-rhel9-moderate:latest
  └── Layer 1: application/vnd.gemara.policy.v1+yaml   (rhel9_policy.yaml)
  └── Layer 2: application/vnd.gemara.catalog.v1+yaml  (rhel9_catalog.yaml)
       Artifact type: application/vnd.gemara.bundle.v1

Verify the bundle is in the registry:

curl -s http://127.0.0.1:5500/v2/nist-800-53-rev5-rhel9-moderate/tags/list
# {"name":"nist-800-53-rev5-rhel9-moderate","tags":["latest"]}

---

Step 4 — Write complytime.yaml

complyctl needs a configuration file pointing it at the registry and telling it:

• Where to find the policy bundle (OCI registry URL)

• Which XCCDF profile to use as the tailoring base

• Which data stream file to use (bypasses OS auto-detection)

# /root/.complytime/complytime.yaml (inside the VM)
policies:
  - url: http://localhost:5500/nist-800-53-rev5-rhel9-moderate
    id: nist-800-53-rev5-rhel9-moderate

targets:
  - id: local
    policies:
      - nist-800-53-rev5-rhel9-moderate
    variables:
      profile: nist_800_53
      datastream: /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

Key gotcha — profile variable constraint: The profile value is validated against ^[a-zA-Z0-9-_.]+$. Use the short name only — do NOT use the full XCCDF ID (xccdf_org.ssgproject.content_profile_nist_800_53).

Key gotcha — datastream: is required: Without this, the OpenSCAP provider reads ID_LIKE from /etc/os-release to pick the data stream, which can resolve to the wrong file. Always set it explicitly to pin the path.

---

Step 5 — complyctl get

Downloads the Policy and ControlCatalog from the OCI registry into the local complyctl workspace.

cd /root/.complytime
complyctl get

What happens:

1. Reads complytime.yaml to find the registry URL

2. Pulls the two-layer OCI bundle via HTTP

3. Writes bundle files into the workspace under .complytime/

Directory after get:

/root/.complytime/
  complytime.yaml
  providers/
    complyctl-provider-openscap
  nist-800-53-rev5-rhel9-moderate/
    rhel9_policy.yaml      # pulled from OCI layer 1
    rhel9_catalog.yaml     # pulled from OCI layer 2

---

Step 6 — complyctl generate

Reads the Policy’s assessment-plans and the XCCDF data stream. Generates a tailored XCCDF profile that selects only the rules listed in the Policy’s assessment-plans.

complyctl generate --policy-id nist-800-53-rev5-rhel9-moderate

What happens:

1. Opens nist-800-53-rev5-rhel9-moderate/rhel9_policy.yaml

2. Extracts all assessment-plans[*].id → these are short CaC rule names

3. Opens build/ssg-rhel9-ds.xml (via the datastream: variable)

4. Finds the base profile xccdf_org.ssgproject.content_profile_nist_800_53

5. Creates a tailoring document that extends the base profile, enabling only the 22 rules from the Policy

Why nist_800_53 as the base profile? The tailoring mechanism uses extend — it starts from nist_800_53 (which selects ALL NIST-mapped rules) and then uses select selected="false" to deselect every rule NOT in the Policy. This is more reliable than enabling rules one by one from an empty base.

Output: A tailored XCCDF XML embedded in the workspace, used by Step 7.

---

Step 7 — complyctl scan

Runs OpenSCAP against the system using the tailored profile from Step 6.

complyctl scan --policy-id nist-800-53-rev5-rhel9-moderate

What happens:

1. Invokes the complyctl-provider-openscap plugin

2. Plugin calls oscap xccdf eval with the tailored profile

3. OpenSCAP evaluates each of the 22 selected rules against the live OS

4. Results are written as ARF (Assessment Results Format) XML

5. complyctl writes a structured evaluation log in YAML

Three output files per baseline:

evaluation-log-nist-800-53-rev5-rhel9-moderate-<timestamp>.yaml

complyctl’s structured summary. Shows Passed/Failed per rule name with the control reference-id for traceability.

evaluations:
- name: configure_custom_crypto_policy_cis
  result: Failed
  control:
    reference-id: nist-800-53-rev5-rhel9-moderate
    entry-id: configure_custom_crypto_policy_cis
  assessment-logs:
  - result: Failed
    start: "2026-06-26T13:12:51Z"
    confidence-level: High

- name: package_sudo_installed
  result: Passed
  control:
    reference-id: nist-800-53-rev5-rhel9-moderate
    entry-id: package_sudo_installed

- name: sudo_add_use_pty
  result: Failed

- name: sudo_remove_no_authenticate
  result: Passed

- name: sudo_remove_nopasswd
  result: Failed

- name: no_invalid_shell_accounts_unlocked
  result: Passed

- name: accounts_tmout
  result: Failed          # ← terminal timeout not configured on fresh VM

- name: sshd_disable_root_login
  result: Failed          # ← root SSH login allowed on fresh VM

- name: kernel_module_usb-storage_disabled
  result: Failed

- name: sysctl_kernel_randomize_va_space
  result: Failed

- name: dir_perms_world_writable_sticky_bits
  result: Passed

- name: file_permissions_unauthorized_world_writable
  result: Passed

- name: file_group_ownership_var_log_audit
  result: Passed

- name: file_permissions_var_log_audit
  result: Passed

# ... (22 total for moderate)

arf.xml — Assessment Results Format

Full OpenSCAP output. Contains per-rule results plus OVAL check details. Parsed with the XCCDF namespace:

python3 << 'EOF'
import xml.etree.ElementTree as ET
ns = {"xccdf": "http://checklists.nist.gov/xccdf/1.2"}
tree = ET.parse("build/complyctl-results/rhel9/moderate/arf.xml")
rules = tree.findall(".//xccdf:rule-result", ns)
summary = {}
for r in rules:
    res = r.find("xccdf:result", ns)
    if res is not None:
        summary[res.text] = summary.get(res.text, 0) + 1
for outcome, count in sorted(summary.items()):
    print(f"  {outcome:25s}: {count}")
EOF
# Output:
#   fail                     : 7
#   notapplicable            : 6
#   notselected              : 1511
#   pass                     : 9

notselected: 1511 — these are the other NIST-mapped rules in the data stream that were deselected by the tailoring. Only 22 rules were actually evaluated.

results.xml — XCCDF benchmark export

The full XCCDF benchmark with the tailored profile embedded, including all rule definitions and their result states. Useful for detailed analysis with oscap report tools.

---

Step 8 — Generate HTML report

Convert the XCCDF results into a human-readable HTML report with rule-level pass/fail details, severity, and rationale.

oscap xccdf generate report \
    build/complyctl-results/rhel9/moderate/results.xml \
    > build/complyctl-results/rhel9/moderate/report.html

Open in browser:

xdg-open build/complyctl-results/rhel9/moderate/report.html
# or
firefox build/complyctl-results/rhel9/moderate/report.html

The report shows:

• Score — percentage of selected rules that passed

• Rule table — each rule with its result (pass/fail/notapplicable), severity, and the XCCDF description of what was checked

• Profile info — which tailored profile was used

Loop for all baselines:

for baseline in low moderate high; do
    oscap xccdf generate report \
        "build/complyctl-results/rhel9/${baseline}/results.xml" \
        > "build/complyctl-results/rhel9/${baseline}/report.html"
    echo "${baseline}: $(wc -c < "build/complyctl-results/rhel9/${baseline}/report.html") bytes"
done

The Ansible scan.yml does this automatically after each baseline scan.

---

Step 9 — Trace results back to NIST controls

Use rules_mapping.yaml to translate a rule PASS/FAIL into a NIST control compliance statement.

python3 << 'EOF'
from ruamel.yaml import YAML
y = YAML()
mapping = y.load(open("build/gemara/rhel9/rules_mapping.yaml"))

# For every rule that failed, find which controls it maps to
failed_rules = [
    "accounts_tmout",
    "sshd_disable_root_login",
    "sudo_add_use_pty",
    "configure_custom_crypto_policy_cis",
    "kernel_module_usb-storage_disabled",
    "sysctl_kernel_randomize_va_space",
    "sudo_remove_nopasswd",
]

print("Failed rules → NIST controls:")
for rule in failed_rules:
    controls = [
        m["source"] for m in mapping["mappings"]
        if any(t["entry-id"] == rule for t in m.get("targets", []))
    ]
    print(f"  {rule}")
    for c in controls:
        print(f"    ← {c.upper()}")
EOF

Expected output:

Failed rules → NIST controls:
  accounts_tmout
    ← AC-2.5
  sshd_disable_root_login
    ← AC-17
    ← AC-17.1
  sudo_add_use_pty
    ← CM-6
  configure_custom_crypto_policy_cis
    ← SC-8
    ← SC-8.1
  kernel_module_usb-storage_disabled
    ← MP-7
  sysctl_kernel_randomize_va_space
    ← SI-16
  sudo_remove_nopasswd
    ← IA-11

---

Summary: file flow diagram

controls/nist_800_53.yml
        │
        └─► Step 2: export_to_gemara.py
                    ├─► build/gemara/rhel9/control_catalog.yaml   (ControlCatalog)
                    ├─► build/gemara/rhel9/rules_mapping.yaml     (MappingDocument)
                    ├─► build/gemara/guidance_catalog.yaml        (GuidanceCatalog, needs OSCAL)
                    └─► products/rhel9/profiles/nist_800_53.profile  (gitignored, Step 1 input)

products/rhel9/profiles/nist_800_53.profile  [generated above]
        │
        └─► Step 1: ./build_product rhel9 -d
                    └─► build/ssg-rhel9-ds.xml  (28 MB, has nist_800_53 XCCDF profile)

build/gemara/rhel9/control_catalog.yaml
        │
        └─► Step 3: generate_complyctl_bundle.py --baseline moderate
                    ├─► build/gemara-bundle/rhel9/moderate/rhel9_policy.yaml   (22 rules)
                    ├─► build/gemara-bundle/rhel9/moderate/rhel9_catalog.yaml  (filtered catalog)
                    └─► [oras push] → registry:5500/nist-800-53-rev5-rhel9-moderate:latest

complytime.yaml  +  registry:5500/...
        │
        ├─► Step 5: complyctl get       → pulls policy + catalog into workspace
        ├─► Step 6: complyctl generate  → creates tailored XCCDF (22 of 1533 rules selected)
        └─► Step 7: complyctl scan      → OpenSCAP evaluates 22 rules against live OS
                    ├─► evaluation-log-*.yaml   (complyctl structured log: Pass/Fail per rule)
                    ├─► arf.xml                 (OpenSCAP Assessment Results Format)
                    └─► results.xml             (XCCDF benchmark with embedded results)

results.xml
        │
        └─► Step 8: oscap xccdf generate report results.xml > report.html
                    └─► report.html   (interactive HTML with rule-level pass/fail + rationale)

evaluation-log-*.yaml  +  build/gemara/rhel9/rules_mapping.yaml
        │
        └─► Step 9: trace rule FAIL → NIST control (AC-2.5, SC-8, IA-11, ...)

---

Running everything with Vagrant (automated)

The Ansible playbooks orchestrate Steps 2–7 on a real RHEL9 VM:

# 1. Start VM (one time)
cd utils/nist_sync/vagrant
vagrant up
bash populate_inventory.sh        # writes ansible/inventory.ini

# 2. Setup VM (one time)
cd ..
ansible-playbook -i ansible/inventory.ini ansible/setup.yml \
    -e complyctl_bin=~/bin/complyctl \
    -e provider_bin=~/.complytime/providers/complyctl-provider-openscap

# 3. Run all three baselines
ansible-playbook -i ansible/inventory.ini ansible/scan.yml

# Results at:
ls build/complyctl-results/rhel9/{low,moderate,high}/