CPU load was not very high, the incoming traffic was not out of the ordinary. So I analyzed the performance metrics of this application. Given that on this box one application was running close to its maximum capacity, I expected that this application could not handle the load during some periods during the day, something we had seen before. It seemed most likely that the applications received more traffic than they could handle. The drop count is shown by the ifconfig command. The packet drops were on incoming traffic. The monitoring system indicated that the packet loss was on a linux box running several server applications that process data sent by our trading machines for analysis by our traders. In this case the network link was reliable fibre, the network link was far from congested. It can be that the network transport is unreliable and packet loss is natural, the network link could be congested, applications cannot handle the offered load. There can be various reasons for packet loss. So I took the alert seriously and started searching for the cause.
However, packet loss is often an early sign that something is wrong with the systems, i.e. The number of lost packets was low and they were TCP packets, so TCP retransmission would overcome the loss. There was no observable degradation of the functionality of the system. The loss was not excessive, only a couple of hundred packets, nothing compared to the billions of packets flowing through our systems. Recently I was alerted by our monitoring system that it detected network packet loss. Hope it’s useful to anyone facing similar issues. Hi all, just wanted to share a recent experience I had investigating a packet drop issue on a linux system. NovemSearching for the cause of dropped packets on Linux.