Home/Resources/Agents/Crypto Library Detection
Cryptography

Crypto Library Detection

Proves FIPS enforcement at the library level with a live MD5 test no version string can fake.

Covered practices · NIST SP 800-171 Rev 2
3.13.11FIPS-validated cryptographyPrimary3.13.10Cryptographic key management3.13.8Transmission confidentiality3.14.1Flaw remediation
4
Crypto stacks audited live (OpenSSL, GnuTLS, NSS, CNG)

The problem

A cryptographic library can be present and FIPS-capable while operating in a non-FIPS mode. OpenSSL 3.x ships with a FIPS provider that has to be explicitly loaded; if the provider list does not include it, every TLS connection and every digest call falls back to the default provider, which still computes MD5 happily and signs the result without protest.

OpenSSL 1.x is end-of-life as of September 2023. Its FIPS module sits on the CMVP Historical list, which means it cannot be cited to claim current FIPS validation under any reading of the standard. Contractors with legacy applications still linking against 1.x discover this when an assessor asks for the certificate number and the contractor cannot produce a current one.

Static linking compounds the problem. An application that ships its own copy of a crypto library bypasses the system configuration entirely. A live MD5 test catches this where a version-only check cannot, because if MD5 succeeds inside the calling process, FIPS enforcement is provably not active no matter what the system policy claims.

What CMMC requires

NIST SP 800-171 Rev 2 controls this agent verifies. Primary mappings drive the gap assessment; secondary mappings provide supporting evidence in the SSP.

Primary mapping
NIST 3.13.11FIPS-validated cryptography

Verifies the libraries actually performing cryptographic operations are FIPS-validated AND running in FIPS mode,including a live algorithm test that catches statically-linked applications.

Read in NIST 800-171 r2
Secondary mapping
NIST 3.13.10Cryptographic key management

Detects libraries that may generate keys with non-approved RNGs, store them in non-approved formats, or permit key sizes below FIPS minimums.

Read in NIST 800-171 r2
NIST 3.13.8Transmission confidentiality

Catches the case where the OS is in FIPS mode but a library used for TLS connections is not,applications can still negotiate non-approved suites through it.

Read in NIST 800-171 r2
NIST 3.14.1Flaw remediation

Treats end-of-life crypto libraries (OpenSSL 1.x) as unpatched flaws requiring upgrade rather than configuration change.

Read in NIST 800-171 r2

How it works

Per-platform detection logic. The agent runs unprivileged where possible and falls back gracefully when raw access is unavailable.

Linux

Runs openssl version -a to identify the major version, then openssl list -providers to confirm the FIPS provider is loaded. Performs a live algorithm test, openssl dgst -md5 /dev/null, and treats success as definitive evidence FIPS enforcement is not active in the calling process. Checks GnuTLS via gnutls-cli --version and the GNUTLS_FORCE_FIPS_MODE environment, and NSS via certutil with a note that FIPS must be enabled per-database with modutil -fips true.

Windows

Notes that Windows CNG is always present and FIPS-validated under cert #4825 for Windows 11/Server 2022 when FipsAlgorithmPolicy is set. Flags applications still using legacy CryptAPI (CAPI), which does not enforce FIPS mode even when the registry policy is on, and surfaces every CAPI-linked binary as a 3.13.11 finding. Builds without an OpenSSL dependency, so the agent itself is FIPS-clean.

macOS

Notes that Apple's Corecrypto is the primary FIPS-validated module on macOS and its status is implied by the OS version check. Surfaces installed third-party OpenSSL builds from Homebrew, MacPorts, or vendored Python distributions and runs the same live MD5 test against each one. Each detected stack is reported with version, install path, and live-test outcome.

What it finds

Concrete findings written to the assessor's mental model,not abstract categories. Severity drives POA&M priority and SPRS deduction.

  • HIGH
    OpenSSL 3.x without FIPS provider loaded
    openssl list -providers does not include the fips entry. Applications linking this OpenSSL can compute MD5, RC4, and other non-approved algorithms without error.
  • HIGH
    OpenSSL 1.x in use
    Library is end-of-life since September 2023. CMVP cert #3819 is on the Historical list,cannot be used to claim current FIPS validation.
  • HIGH
    Live MD5 digest succeeded
    openssl dgst -md5 returned a hash. FIPS enforcement is provably not active in this process, regardless of what the library version reports.
  • MEDIUM
    NSS database not in FIPS mode
    certutil is present but no NSS database has been initialized with modutil -fips true. Java applications using SunPKCS11 inherit non-FIPS behavior.
  • LOW
    Legacy CAPI usage detected
    Application still links against the legacy CryptAPI. CAPI does not enforce FIPS mode even when the registry policy is on,Windows applications can bypass FipsAlgorithmPolicy through this path.
SPRS impact

Library-level FIPS gaps map to 3.13.11, a 5-point control, the same way OS-level gaps do, but they are substantially harder to catch by hand. A system can pass the OS FIPS check and still fail this one because of a vendored library inside a packaged application or a forgotten OpenSSL 1.x install. The live MD5 test is the only check that conclusively proves enforcement at the process level, and it runs in milliseconds. End-of-life OpenSSL 1.x maps to 3.14.1 as well, an additional 3-point exposure that compounds the deduction.

How it hooks into the platform

The platform finds every host with installed crypto libraries during auto-discovery and runs the live MD5 test plus provider enumeration on its own cadence. You decide whether the inventory and live-test results stay on-prem or sync up. The catalog feeds the SSP's component list with every library, version, and FIPS status. Findings link to 3.13.11 evidence with the live MD5 outcome attached. POA&M items open for every OpenSSL 1.x install with the upgrade path identified, and regressions surface when a container ships a vendored copy.

How we know it's working

Acceptance criteria from the engineering spec,what the agent must do to ship.

  • Runs a live MD5 algorithm test to prove process-level FIPS enforcement
  • Audits each library independently so one failure cannot mask another
  • Ships without an OpenSSL dependency on Windows, FIPS-clean by build
  • Detects GnuTLS FIPS-mode environment overrides per process
  • Returns the full library catalog in structured output, even on clean hosts
Other agents in Cryptography

Run this agent against your environment.

Book a 30-minute scoping call. We'll deploy the agent on your systems and walk through the findings together.

Book a Scoping Call →