Signing large files with PHP openssl extension

The openssl extension for PHP provides the openssl_sign() function to sign data.

The drawback of this function is that the data is fed with a single variable holding the entire data to be signed. This implies that all your data must fit in memory. This is inadequate if you want to sign large files and cannot/dont want to load the whole file in memory with a get_file_contents() for example.

Here is a way to circumvent this limitation by implementing your own signing process (bonus point for being compatible with the output from the original openssl_sign()/openssl_verify() functions).

Basically, signing a file consist in:

1. Compute a digest of the file with a hashing function like SHA1
2. Encrypt this digest with your private key

This gives you a signature data that you can be verified by doing the reverse:

1. Decrypt the signature with the signer public key
2. Compute a digest of the file with the same hashing function used by the signer
3. Compare your digest with the decrypted one. If they are  different, then your file is not the original one

So, based on this principles, here are a sign and verify function that do not requires you to load the entire file into memory:

• sign_file()

$privKeyfile = '/path/to/your/private_key.pem';

function sign_file($privKeyfile, $file) {
  $digest = sha1_file($file, true);

  $privkey = openssl_get_privatekey(file_get_contents($privKeyfile));
  openssl_private_encrypt($digest, $signature, $privkey);

  return $signature;
}

$signature = sign_file($privKeyfile, 'large_file.iso');

• verify_file()

$pubKeyfile = '/path/to/your/public_key.pem';

function verify_file($pubKeyfile, $file, $signature) {
  $digest = sha1_file($file, true);

  $pubkey = openssl_get_publickey(file_get_contents($pubKeyfile));
  openssl_public_decrypt($signature, $decrypted_digest, $pubkey);

  if( $digest == $decrypted_digest ) {
    return true;
  }

  return false;
}

$isValid = verify_file($pubKeyfile, 'large_file.iso', $signature);
if( $isValid ) {
  print "Valid signature.\n";
} else {
  print "Invalid signature!\n";
}

But there is still something wrong (the bonus point).

If you generate a signature with sign_file() and try to verify the signature with openssl_verify() you’ll notice that it will fail. The same happens if you sign with openssl_sign() and verify with verify_file().

Despite the fact that openssl_sign() seems to do the same operations (compute digest, then encrypt the digest with the priv key), the resulting signature is not the same.

Afer grepping into the openssl source code for a while, I finally found that openssl does not encrypt the « raw » digest, but encrypt the digest in its ASN1 form.

So, if you want to generate, or verify an openssl signature, you’ll have to also encode/decode your digests in ASN1.

Here is a rewrite of the sign_file() and verify_file() functions that is compatible with the signature from openssl_sign() and openssl_verify() :

• sign_file() with « hardcoded » ASN1 encoding

$privKeyfile = '/path/to/your/private_key.pem';

function sign_file($privKeyfile, $file) {
  $digest = sha1_file($file, true);

  $asn1  = chr(0x30).chr(0x21); // SEQUENCE, 33
  $asn1 .= chr(0x30).chr(0x09); // SEQUENCE, 9
  $asn1 .= chr(0x06).chr(0x05); // OBJECT IDENTIFIER, 5
  $asn1 .= chr(0x2b).chr(0x0e).chr(0x03).chr(0x02).chr(0x1a); // 1.3.14.3.2.26 (SHA1)
  $asn1 .= chr(0x05).chr(0x00); // NULL
  $asn1 .= chr(0x04).chr(0x14); // OCTET STRING, 20
  $asn1 .= $digest;

  $privkey = openssl_get_privatekey(file_get_contents($privKeyfile));
  openssl_private_encrypt($asn1, $signature, $privkey);

  return $signature;
}

$signature = sign_file($privKeyfile, 'large_file.iso');

• verify_file() with « hardcoded » ASN1 decoding

$pubKeyfile = '/path/to/your/public_key.pem';

function verify_file($pubKeyfile, $file, $signature) {
  $digest = sha1_file($file, true);

  $pubkey = openssl_get_publickey(file_get_contents($pubKeyfile));
  openssl_public_decrypt($signature, $asn1, $pubkey);

  $decrypted_digest = substr($asn1, 15); // Blindly strip the ASN1 header

  if( $digest == $decrypted_digest ) {
    return true;
  }

  return false;
}

$isValid = verify_file($pubKeyfile, 'large_file.iso', $signature);
if( $isValid ) {
  print "Valid signature.\n";
} else {
  print "Invalid signature!\n";
}

Now, you can sign with openssl_sign() and verify with verify_file(), or sign with sign_file() and verify with openssl_verify().