FruUtils: Rework calculateChecksum() implementation
FRU areas are sized by a single-byte count field in units of FRU blocks,
which are 8 bytes in size. As such, it's possible for a FRU area to be
up to 255 * 8 or 2040 bytes in size. An accumulation of values over such
an area to calculate the checksum can produce values exceeding 256. In
such a circumstance the current checksum implementation will mask a
negative value, invoking undefined behaviour.
Instead, accumulate into an unsigned, 8-bit type to avoid negative
masking through well-defined wrap behaviour.
Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Change-Id: Ie85894004f1048413cddc62d68dd0b7ae8f6f541
diff --git a/src/FruUtils.cpp b/src/FruUtils.cpp
index 945abf4..561b007 100644
--- a/src/FruUtils.cpp
+++ b/src/FruUtils.cpp
@@ -464,10 +464,8 @@
std::vector<uint8_t>::const_iterator end)
{
constexpr int checksumMod = 256;
- constexpr uint8_t modVal = 0xFF;
- int sum = std::accumulate(iter, end, 0);
- int checksum = (checksumMod - sum) & modVal;
- return static_cast<uint8_t>(checksum);
+ uint8_t sum = std::accumulate(iter, end, static_cast<uint8_t>(0));
+ return (checksumMod - sum) % checksumMod;
}
uint8_t calculateChecksum(std::vector<uint8_t>& fruAreaData)