ACVP Client

ACVP is the next version of NIST's CAVP—a program for running cryptographic implementations against a set of test vectors. CAVP involved emailing around zip files of somewhat-INI-like test vectors where no two files had quite the same format. ACVP is supposed to replace that with a) TLS connections rather than email (but not yet) and b) JSON rather than bespoke formats.

The tool in this directory can speak to ACVP servers, fetch test vectors, and lower the operations to a simpler protocol. A wrapper around the FIPS module in question implements that protocol for testing. Test vectors can also come from files on disk.

The tool also provides an interface for querying and manipulating the ACVP database, which contains lists of modules, vendors, contacts, operating environments, etc.

For an alternative, see libacvp.

Building

This tool builds independently of the rest of the BoringSSL: just run go build within the acvptool directory.

File-based operation

While ACVP hopes to replace emailing files around at some point, emailing is still common practice with most NVLAP labs. To process a file of test vectors, use the -json flag:

% ./acvptool -json input.json > output.json

The top-level structure of these JSON files is not specified by NIST. This tool consumes the form that appears to be most commonly used.

The lab will need to know the configuration of the module to generate tests. Obtain that with the -regcap option and redirect the output to a file.

Testing other FIPS modules

Lowering ACVP to a simpler form might be useful for other modules so the protocol is described here. The tool has far from complete coverage of ACVP‘s mountain of options, but common cases are handled. If you have additional needs then it’s hopefully straightforward to extend the tool yourself.

The FIPS module being tested needs to be wrapped such that the tool can fork and exec a binary that speaks this protocol over stdin/stdout. For BoringSSL that binary is in the modulewrapper directory and serves as a reference implementation if you have questions about the protocol that aren‘t answered below. BoringSSL’s modulewrapper contains the FIPS module itself, but your binary could forward the communication over, e.g., a serial link to a hardware module. Specify the path to the binary with the -wrapper option.

The protocol is request–response: the subprocess only speaks in response to a request and there is exactly one response for every request. Requests consist of one or more byte strings and responses consist of zero or more byte strings.

A request contains: the number of byte strings, the length of each byte string, and the contents of each byte string. All numbers are 32-bit little-endian and values are concatenated in the order specified. The first byte string is mandatory and is the name of the command to perform. A response has the same format except that there may be zero byte strings and the first byte string has no special meaning.

All implementations must support the getConfig command which takes no arguments and returns a single byte string which is a JSON blob of ACVP algorithm configuration. This blob describes all the algorithms and capabilities that the module supports and is an array of JSON objects suitable for including as the algorithms value when creating an ACVP vector set.

Crafting this JSON is an art. You can get some information from reading the NIST documentation but you might also want to crib from the BoringSSL wrapper in modulewrapper.

The other commands are as follows. (Note that you only need to implement the commands required by the ACVP configuration returned.)

CommandArgumentsOutputs
3DES-CBC/decryptKey, ciphertext, IV, num iterations¹Result, Previous result
3DES-CBC/encryptKey, plaintext, IV, num iterations¹Result, Previous result
3DES/decryptKey, input block, num iterations¹Result, Previous result
3DES/encryptKey, input block, num iterations¹Result, Previous result
AES-CBC/decryptKey, ciphertext, IV, num iterations¹Result, Previous result
AES-CBC/encryptKey, plaintext, IV, num iterations¹Result, Previous result
AES-CBC-CS3/decryptKey, ciphertext, IV, num iterations²Result
AES-CBC-CS3/encryptKey, plaintext, IV, num iterations²Result
AES-CCM/openTag length, key, ciphertext, nonce, adOne-byte success flag, plaintext or empty
AES-CCM/sealTag length, key, plaintext, nonce, adCiphertext
AES-CTR/decryptKey, ciphertext, initial counter, constant 1Plaintext
AES-CTR/encryptKey, plaintexttext, initial counter, constant 1Ciphertext
AES-GCM/openTag length, key, ciphertext, nonce, adOne-byte success flag, plaintext or empty
AES-GCM/sealTag length, key, plaintext, nonce, adCiphertext
AES-KW/open(dummy), key, ciphertext, (dummy), (dummy)One-byte success flag, plaintext or empty
AES-KW/seal(dummy), key, plaintext, (dummy), (dummy)Ciphertext
AES-KWP/open(dummy), key, ciphertext, (dummy), (dummy)One-byte success flag, plaintext or empty
AES-KWP/seal(dummy), key, plaintext, (dummy), (dummy)Ciphertext
AES-XTS/decryptKey, ciphertext, tweakPlaintext
AES-XTS/encryptKey, plaintext, tweakCiphertext
AES/decryptKey, input block, num iterations¹Result, Previous result
AES/encryptKey, input block, num iterations¹Result, Previous result
CMAC-AESNumber output bytes, key, messageMAC
CMAC-AES/verifyKey, message, claimed MACOne-byte success flag
ctrDRBG/AES-256Output length, entropy, personalisation, ad1, ad2, nonceOutput
ctrDRBG-reseed/AES-256Output length, entropy, personalisation, reseedAD, reseedEntropy, ad1, ad2, nonceOutput
ctrDRBG-pr/AES-256Output length, entropy, personalisation, ad1, entropy1, ad2, entropy2, nonceOutput
ECDH/<CURVE>X, Y, private keyX, Y, shared key
ECDSA/keyGenCurve namePrivate key, X, Y
ECDSA/keyVerCurve name, X, YSingle-byte valid flag
ECDSA/sigGenCurve name, private key, hash name, messageR, S
ECDSA/sigVerCurve name, hash name, message, X, Y, R, SSingle-byte validity flag
FFDHp, q, g, peer public key, local private key (or empty), local public key (or empty)Local public key, shared key
HKDF/<HASH>key, salt, info, num output bytesKey
HMAC-SHA-1Value to hash, keyDigest
HMAC-SHA2-224Value to hash, keyDigest
HMAC-SHA2-256Value to hash, keyDigest
HMAC-SHA2-384Value to hash, keyDigest
HMAC-SHA2-512Value to hash, keyDigest
HMAC-SHA2-512/256Value to hash, keyDigest
hmacDRBG/<HASH>Output length, entropy, personalisation, ad1, ad2, nonceOutput
hmacDRBG-reseed/<HASH>Output length, entropy, personalisation, reseedAD, reseedEntropy, ad1, ad2, nonceOutput
hmacDRBG-pr/<HASH>Output length, entropy, personalisation, ad1, entropy1, ad2, entropy2, nonceOutput
KDF-counterNumber output bytes, PRF name, counter location string, key, number of counter bitsCounter, output
RSA/keyGenModulus bit-sizee, p, q, n, d
RSA/sigGen/<HASH>/pkcs1v1.5Modulus bit-sizen, e, signature
RSA/sigGen/<HASH>/pssModulus bit-sizen, e, signature
RSA/sigVer/<HASH>/pkcs1v1.5n, e, message, signatureSingle-byte validity flag
RSA/sigVer/<HASH>/pssn, e, message, signatureSingle-byte validity flag
SHA-1Value to hashDigest
SHA2-224Value to hashDigest
SHA2-256Value to hashDigest
SHA2-384Value to hashDigest
SHA2-512Value to hashDigest
SHA2-512/256Value to hashDigest
SHA-1/MCTInitial seed¹Digest
SHA2-224/MCTInitial seed¹Digest
SHA2-256/MCTInitial seed¹Digest
SHA2-384/MCTInitial seed¹Digest
SHA2-512/MCTInitial seed¹Digest
SHA2-512/256/MCTInitial seed¹Digest
TLSKDF/<1.0|1.2>/<HASH>Number output bytes, secret, label, seed1, seed2Output

¹ The iterated tests would result in excessive numbers of round trips if the module wrapper handled only basic operations. Thus some ACVP logic is pushed down for these tests so that the inner loop can be handled locally. Either read the NIST documentation (block-ciphers hashes) to understand the iteration count and return values or, probably more fruitfully, see how these functions are handled in the modulewrapper directory.

² Will always be one because MCT tests are not supported for CS3.

Online operation

If you have credentials to speak to either of the NIST ACVP servers then you can run the tool in online mode.

Configuration is done via a config.json file in the current working directory. Here's a template:

{
        "ACVPServer": "https://demo.acvts.nist.gov/",
        "CertPEMFile": "certificate_from_nist.pem",
        "PrivateKeyFile": "your_private_key.key",
        "TOTPSecret": "<base64 from NIST goes here>",
        "SessionTokensCache": "~/.cache/acvp-session-tokens",
        "LogFile": "log"
}

NIST‘s ACVP servers use both TLS client certificates and TOTP for authentication. When registering with NIST they’ll sign a CSR and return a certificate in PEM format, which is pointed to by CertPEMFile. The corresponding private key is expected in PrivateKeyFile. Lastly, NIST will provide a file that contains the base64-encoded TOTP seed, which must be pasted in as the value of TOTPSecret.

NIST's ACVP server provides special access tokens for each test session and test sessions can only be accessed via those tokens. The reasoning behind this is unclear but this client can, optionally, keep records of these access tokens in the directory named by SessionTokensCache. If that directory name begins with ~/ then that prefix will be replaced with the value of $HOME.

Lastly, a log of all HTTP traffic will be written to the file named by LogFile, if provided. This is useful for debugging.

Interactive Use

ACVP provides a fairly complex interface to a database of several types of objects. A rough UI is provided for this which is triggered when the client is invoked with no command-line arguments.

The simplest objects in ACVP are request objects. These record the status of requested changes to the database and, in practice, changes to the NIST demo database never succeed. The set of pending requests for the current user can be enumerated just by evaluating the requests object:

> requests
[
  {
    "url": "/acvp/v1/requests/374",
    "status": "processing"
  },
  {
    "url": "/acvp/v1/requests/218",
    "status": "processing"
  }
]

A specific request can be evaluated by using indexing syntax:

> requests[374]
{
  "url": "/acvp/v1/requests/374",
  "status": "processing"
}

The list of vendors provides a more complex example. Since there are large number of duplicates in NIST‘s database, there are more than 10 000 vendor objects and enumerating them all takes a long time. Thus evaluating the vendors object doesn’t do that:

> vendors
[object set vendors]

It is still possible to use indexing syntax to read a specific vendor object if you know the ID:

> vendors[1234]
{
  "url": "/acvp/v1/vendors/1234",
  "name": "Apple Inc.",
  "website": "www.apple.com",
  "contactsUrl": "/acvp/v1/vendors/1234/contacts",
  "addresses": [
    {
      "url": "/acvp/v1/vendors/1234/addresses/1234",
      "street1": "1 Infinite Loop",
      "locality": "Cupertino",
      "region": "CA",
      "country": "USA",
      "postalCode": "95014"
    }
  ]
}

Finding a vendor when the ID is not known requires searching and the ACVP spec documents, for each object type, what values and what relations can be searched on. This is reflected in a variant of the indexing syntax:

> vendors[where name contains "Google LLC"]
[
  {
    "url": "/acvp/v1/vendors/11136",
    "name": "Google LLC",
    "website": "www.google.com",
    "contactsUrl": "/acvp/v1/vendors/11136/contacts",
    "addresses": [
      {
        "url": "/acvp/v1/vendors/11136/addresses/11136",
        "street1": "1600 Amphitheatre Parkway",
        "locality": "Mountain View",
        "region": "CA",
        "country": "USA",
        "postalCode": "94043"
      }
    ]
  },
  {
    "url": "/acvp/v1/vendors/11137",
    "name": "Google LLC",
    "website": "www.google.com",
    "contactsUrl": "/acvp/v1/vendors/11137/contacts",
    "addresses": [
      {
        "url": "/acvp/v1/vendors/11137/addresses/11137",
        "street1": "1600 Amphitheatre Parkway",
        "locality": "Mountain View",
        "region": "CA",
        "country": "USA",
        "postalCode": "94043"
      }
    ]
  }
]

In general, && and || can be used as in C and the relationships are ==, !=, contains, startsWith, and endsWith. Only values and relations listed in the ACVP spec for a given object can be used.

More complex interaction remains to be fleshed out. However, it is generally possible to create new objects by evaluating, for example, vendors.new(). That will invoke $EDITOR to edit the JSON to be submitted. (For now, however, no helpful templates are provided.)

The current list of objects is:

  • requests
  • vendors
  • persons
  • modules
  • oes (operating environments)
  • deps
  • algos
  • sessions

Running test sessions

In online mode, a given algorithm can be run by using the -run option. For example, -run SHA2-256. This will fetch a vector set, have the module-under-test answer it, and upload the answer. If you want to just fetch the vector set for later use with the -json option (documented above) then you can use -fetch instead of -run. The -fetch option also supports passing -expected-out <filename> to fetch and write the expected results, if the server supports that.

After results have been produced with -json, they can be uploaded with -upload. So -run is effectively these three steps combined:

./acvptool -fetch SHA2-256 > request
./acvptool -json request > result
./acvptool -upload result