tree 0b65956b82fedccbcb05f8f196c3726da317c399
parent 13eba41aaba1e704afe63db23e287518c1ae6dff
author Patrick Williams <patrick@stwcx.xyz> 1638707544 -0600
committer Patrick Williams <patrick@stwcx.xyz> 1638708828 -0600

certificate: fix memory leak

When the certificate test cases were ran under ASAN, there was reported
to be a large memory leak in the certificate code.

The understanding documented previously in the code with respect to the
relationship between `X509_STORE` and `X509_LOOKUP` did not match the
reality of the OpenSSL source.  `LOOKUP` is part-of the `STORE`, not an
owner-of it.  It is not appropriate to `X509_LOOKUP_free` the `LOOKUP`
that has become part of the `STORE`, because it is the responsibility of
the `STORE` to do that.

Invert the relationship (and holding std::unique_ptrs) so that the
`X509_STORE` becomes the RAII object and everything contained in it can
be freed when it goes out of scope.

Further explanation of the OpenSSL source is as follows:

* The `X509_LOOKUP_free` only releases the memory held by itself and
  calls `method->free`[1].  The `X509_LOOKUP_file` type has no
  `method->free`[2] (and confirmed with GDB).  This means that the
  `X509_LOOKUP_free` does not end up freeing much of any memory and
  causes a leak of everything it put into the `X509_STORE`, so
  `X509_LOOKUP_free` does not belong as the RAII cleanup function.

* The `X509_STORE_add_lookup` allocates the `X509_LOOKUP`, assigns
  itself to the `LOOKUP` as the store context, and adds the `LOOKUP`
  onto a stack of held `LOOKUP` objects[3].  When `X509_STORE_free` is
  called, all of the held `LOOKUP` objects are freed[4] by calling
  `X509_LOOKUP_free`.

Therefore, assigning `X509_STORE_free` as the RAII cleanup function
allows both the `X509_STORE` (and all certificate data inserted into it)
as well as the created `X509_LOOKUP` to be freed.

Tested: ASAN now passes, along with testcases, when ran.

1. https://github.com/openssl/openssl/blob/master/crypto/x509/x509_lu.c#L39
2. https://github.com/openssl/openssl/blob/master/crypto/x509/by_file.c#L30
3. https://github.com/openssl/openssl/blob/master/crypto/x509/x509_lu.c#L285
4. https://github.com/openssl/openssl/blob/master/crypto/x509/x509_lu.c#L238

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ie350c47a2c01c5a47ed538d01e1f71274ece1fc8