[PATCH 2/3] test/security_inline_proto: fix MTU calculation underflow

[Bruce Richardson]

When testing with some NICs it was observed that the max MTU calculation
could underflow. Despite the RTE_MIN, due to integer promotion, this
lead to the configuring of the port with an excessively large, invalid
value. For example, if lim_nb_seg_max == 0, that meant that
max_data_room - 256 is -256 for comparison purposes using standard
integers (promoted from uint16_t). That is lower than the max mtu so the
-256 value is chosen, which becomes >65000 when converted back to
uint16_t.

Fix this by a) checking for seg_max == 0 b) doing calculations using
uint32_t and c) checking explicitly for underflow before subtracting.

Fixes: 3edd1197a605 ("test/security: add multi-segment inline IPsec cases")
Cc: stable at dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson at intel.com>
---
 app/test/test_security_inline_proto.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/app/test/test_security_inline_proto.c b/app/test/test_security_inline_proto.c
index bc3ef54f71..81fce7364c 100644
--- a/app/test/test_security_inline_proto.c
+++ b/app/test/test_security_inline_proto.c
@@ -2045,8 +2045,19 @@ inline_ipsec_testsuite_setup(void)
 	memcpy(&local_port_conf, &port_conf, sizeof(port_conf));
 	/* Add Multi seg flags */
 	if (sg_mode) {
-		uint16_t max_data_room = RTE_MBUF_DEFAULT_DATAROOM *
-			dev_info.rx_desc_lim.nb_seg_max;
+		uint32_t max_data_room;
+
+		if (dev_info.rx_desc_lim.nb_seg_max == 0) {
+			printf("SG mode unsupported: invalid max Rx segments (0)\n");
+			return TEST_SKIPPED;
+		}
+
+		max_data_room = RTE_MBUF_DEFAULT_DATAROOM * dev_info.rx_desc_lim.nb_seg_max;
+		if (max_data_room <= 256) {
+			printf("SG mode unsupported: max data room (%u) too small\n",
+				max_data_room);
+			return TEST_SKIPPED;
+		}
 
 		local_port_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_SCATTER;
 		local_port_conf.txmode.offloads |= RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
-- 
2.53.0