Fix DOS attack scenario
Problem : When 201 connections made in parallel to BMC and closed them
immediately without properly sending a close_notify alert it was
observed that Bmcweb server was taking several minutes to close the
sockets. All the 200 TCP sockets were in CLOSE_WAIT state.
Journal log shows below line
[CRITICAL http_connection.hpp:213] 0x29d1ef0Max connection count
exceeded.
```
Not able to make new connection
$ curl -k -H "X-Auth-Token:$bmc_token" -X GET https://${BMC_IP}/redfish
/v1/AccountService/Accounts
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to
127.0.0.1:2443
Fix : The bmcweb server failed to identify the end of stream at TCP (SSL
/TLS) layer, therefore added check to identify the end of stream which
closes the connection and socket
Test :
I tested this MR using a script that simulated 5,000 parallel
connections simultaneously to BMC and closed them immediately without
properly sending a close_notify alert
Observations:
The BMC became unresponsive for 30-40 seconds before recovering.
After recovery, it took approximately 90 seconds to close all
connections in QEMU.
On real hardware, connection closure times may be slightly higher
(though still within expected parameters).
Conclusion:
This behavior aligns with expectations.
After 90 seconds observed that
1) No sockets in CLOSE_WAIT state
2) Able to make new connection.
curl -k -H "X-Auth-Token:$bmc_token" https://${IP}/redfish/v1/AccountService/Accounts
{
"@odata.id": "/redfish/v1/AccountService/Accounts",
"@odata.type": "#ManagerAccountCollection.ManagerAccountCollection",
"Description": "BMC User Accounts",
"Members": [
{
"@odata.id": "/redfish/v1/AccountService/Accounts/root"
}
],
"Members@odata.count": 1,
"Name": "Accounts Collection"
}
```
Change-Id: I1c277db0b774d33c656b4a2b1bd14f3575535bec
Signed-off-by: Chandramohan Harkude <chandramohan.harkude@gmail.com>
diff --git a/http/http_connection.hpp b/http/http_connection.hpp
index 955cb34..b3d803a 100644
--- a/http/http_connection.hpp
+++ b/http/http_connection.hpp
@@ -21,6 +21,7 @@
#include <boost/asio/error.hpp>
#include <boost/asio/ip/tcp.hpp>
+#include <boost/asio/ssl/error.hpp>
#include <boost/asio/ssl/stream.hpp>
#include <boost/asio/ssl/stream_base.hpp>
#include <boost/asio/ssl/verify_context.hpp>
@@ -610,7 +611,8 @@
doWrite();
return;
}
- if (ec == boost::beast::http::error::end_of_stream)
+ if (ec == boost::beast::http::error::end_of_stream ||
+ ec == boost::asio::ssl::error::stream_truncated)
{
BMCWEB_LOG_WARNING("{} End of stream, closing {}", logPtr(this),
ec);
@@ -715,6 +717,15 @@
return;
}
+ if (ec == boost::beast::http::error::end_of_stream ||
+ ec == boost::asio::ssl::error::stream_truncated)
+ {
+ BMCWEB_LOG_WARNING("{} End of stream, closing {}", logPtr(this),
+ ec);
+ hardClose();
+ return;
+ }
+
gracefulClose();
return;
}
@@ -783,6 +794,16 @@
doWrite();
return;
}
+
+ if (ec == boost::beast::http::error::end_of_stream ||
+ ec == boost::asio::ssl::error::stream_truncated)
+ {
+ BMCWEB_LOG_WARNING("{} End of stream, closing {}", logPtr(this),
+ ec);
+ hardClose();
+ return;
+ }
+
if (ec)
{
BMCWEB_LOG_DEBUG("{} from write(2)", logPtr(this));