sensor: Implement sensor "ASYNC_READ_TIMEOUT"
This change will prevent sensors from blocking all other sensor reads
and D-Bus if they do not report failures quickly enough.
If "ASYNC_READ_TIMEOUT" environment variable is defined in the
sensor's config file for a key_type, the sensor read will be
asynchronous with timeout set in milliseconds.
For example for "sensor1":
ASYNC_READ_TIMEOUT_sensor1="1000" // Timeout will be set to 1 sec
If the read times out, the sensor read will be skipped and the
sensor's functional property will be set to 'false'. Timed out futures
will be placed in a map to prevent their destructor from running and
blocking until the read completes (limitation of std::async).
Tested: This patch has been running downstream for over a year to
solve a slow I2C sensor reads causing IPMI slowdown.
Change-Id: I3d9ed4d5c9cc87d3196fc281451834f3001d0b48
Signed-off-by: Brandon Kim <brandonkim@google.com>
diff --git a/sensor.hpp b/sensor.hpp
index 4b2d281..a1e0524 100644
--- a/sensor.hpp
+++ b/sensor.hpp
@@ -4,7 +4,10 @@
#include "sensorset.hpp"
#include "types.hpp"
+#include <cerrno>
+#include <future>
#include <gpioplus/handle.hpp>
+#include <map>
#include <memory>
#include <optional>
#include <stdplus/handle/managed.hpp>
@@ -13,6 +16,8 @@
namespace sensor
{
+using TimedoutMap = std::map<SensorSet::key_type, std::future<int64_t>>;
+
struct valueAdjust
{
double gain = 1.0;
@@ -20,6 +25,17 @@
std::unordered_set<int> rmRCs;
};
+/** @brief Custom exception for async sensor reading timeout
+ */
+struct AsyncSensorReadTimeOut : public std::system_error
+{
+ AsyncSensorReadTimeOut() :
+ system_error(std::error_code(ETIMEDOUT, std::system_category()),
+ "Async sensor read timed out")
+ {
+ }
+};
+
/** @class Sensor
* @brief Sensor object based on a SensorSet container's key type
* @details Sensor object to create and modify an associated device's sensor
@@ -87,10 +103,13 @@
* (number of and delay between)
* @param[in] info - Sensor object information
*
+ * @param[in] timedoutMap - Map to track timed out threads
+ *
* @return - Shared pointer to the value object
*/
std::shared_ptr<ValueObject> addValue(const RetryIO& retryIO,
- ObjectInfo& info);
+ ObjectInfo& info,
+ TimedoutMap& timedoutMap);
/**
* @brief Add status interface and functional property for sensor
@@ -177,4 +196,29 @@
*/
std::optional<GpioLocker> gpioUnlock(const gpioplus::HandleInterface* handle);
+/**
+ * @brief Asynchronously read a sensor with timeout defined by
+ * ASYNC_READ_TIMEOUT environment variable
+ *
+ * @param[in] sensorSetKey - Sensor object's identifiers
+ * @param[in] ioAccess - Hwmon sysfs access
+ * @param[in] asyncTimeout - Async read timeout in milliseconds
+ * @param[in] timedoutMap - Map to track timed out threads
+ *
+ * (Params needed for HwmonIO::read)
+ * @param[in] type - The hwmon type (ex. temp).
+ * @param[in] id - The hwmon id (ex. 1).
+ * @param[in] sensor - The hwmon sensor (ex. input).
+ * @param[in] retries - The number of times to retry.
+ * @param[in] delay - The time to sleep between retry attempts.
+ *
+ * @return - SensorValueType read asynchronously, will throw if timed out
+ */
+SensorValueType asyncRead(const SensorSet::key_type& sensorSetKey,
+ const hwmonio::HwmonIOInterface* ioAccess,
+ std::chrono::milliseconds asyncTimeout,
+ TimedoutMap& timedoutMap, const std::string& type,
+ const std::string& id, const std::string& sensor,
+ const size_t retries,
+ const std::chrono::milliseconds delay);
} // namespace sensor