TCP retransmission

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

TCP retransmission

Nikolas Karakotas-3
Hi,

I'm facing an issue on-site where there is quite a lot of traffic.
Sometimes I can see from the log that we have a TCP re-transmission
and we once the packet is re transmitted the data we receive is
impartial. As a result our modbus driver responds with and error.
I can see that the the modbus header is partial correct but the rest
is junk. This is very difficult to reproduce but some insight may help
me to find the issue. So do you believe it's an issue with the:
1. Ethernet driver
2. Application layer dealing with fragmented packet



Nikolas

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

tcp-retransmit.PNG (179K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: TCP retransmission

goldsimon@gmx.de
Nikolas Karakotas wrote:
> I'm facing an issue on-site where there is quite a lot of traffic.
> Sometimes I can see from the log that we have a TCP re-transmission
> and we once the packet is re transmitted the data we receive is
> impartial. As a result our modbus driver responds with and error.
> I can see that the the modbus header is partial correct but the rest
> is junk. This is very difficult to reproduce but some insight may help
> me to find the issue. So do you believe it's an issue with the:

If you really want help, you'll at least have to attach a packet trace
file, not some image.
Also, explain which communication we see in that file (explain IPs,
connections, applications, etc).
Plus, tell us exactly which packet you seem is wrong. All that makes it
much easier to have a quick look and maybe see something. From the
picture you sent, I can see just about nothing.

> 1. Ethernet driver
> 2. Application layer dealing with fragmented packet

Ah, see, both can be wrong, I guess. And the lwIP port can be bad as
well. How could we know without even knowing what you are using, where
it comes from, etc.
Share some more input and maybe someone here can help you.

Simon

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: TCP retransmission

Nikolas Karakotas-3
Hi,

True for not providing enough information.

Deducing wireshark information I can say:
1. Remote host send a Modbus request
2. For whichever reason we don't respond to it as it may have been
lost because of network or Ethernet driver
3. The remote host then re sends the request and we are able to
receive it notifying with a Dup Ack
4. The remote host then sends the re-transmission of the previous
packets lost with 24 bytes ( contains 2 modbus requests now)
5. It's forwarded to the application layer but I found out that the
application layer Modbus implementation doesn't handle fragmented
Modbus requests well

I'm currently fixing this by using the captured 24 bytes Modbus request.
How can I test such scenario to find out where the issue is and I'm
missing packets? We have used tools to flood the controller etc but
never show such issue.
The loss may happen once a day but signals and alarm to the SCADA
system and isn't acceptable.

Attached a snapshot.

Nikolas



On Tue, Nov 21, 2017 at 7:39 PM, [hidden email] <[hidden email]> wrote:

> Nikolas Karakotas wrote:
>>
>> I'm facing an issue on-site where there is quite a lot of traffic.
>> Sometimes I can see from the log that we have a TCP re-transmission
>> and we once the packet is re transmitted the data we receive is
>> impartial. As a result our modbus driver responds with and error.
>> I can see that the the modbus header is partial correct but the rest
>> is junk. This is very difficult to reproduce but some insight may help
>> me to find the issue. So do you believe it's an issue with the:
>
>
> If you really want help, you'll at least have to attach a packet trace file,
> not some image.
> Also, explain which communication we see in that file (explain IPs,
> connections, applications, etc).
> Plus, tell us exactly which packet you seem is wrong. All that makes it much
> easier to have a quick look and maybe see something. From the picture you
> sent, I can see just about nothing.
>
>> 1. Ethernet driver
>> 2. Application layer dealing with fragmented packet
>
>
> Ah, see, both can be wrong, I guess. And the lwIP port can be bad as well.
> How could we know without even knowing what you are using, where it comes
> from, etc.
> Share some more input and maybe someone here can help you.
>
> Simon
>
> _______________________________________________
> lwip-users mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/lwip-users

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

dup-ack.pcapng (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: TCP retransmission

goldsimon@gmx.de
Nikolas Karakotas wrote:

> True for not providing enough information.
>
> Deducing wireshark information I can say:
> 1. Remote host send a Modbus request
> 2. For whichever reason we don't respond to it as it may have been
> lost because of network or Ethernet driver
> 3. The remote host then re sends the request and we are able to
> receive it notifying with a Dup Ack
> 4. The remote host then sends the re-transmission of the previous
> packets lost with 24 bytes ( contains 2 modbus requests now)
> 5. It's forwarded to the application layer but I found out that the
> application layer Modbus implementation doesn't handle fragmented
> Modbus requests well

Please, more info! I can only assume that lwIP is the Modbus-TCP server
having the IP address 10.200.61.1 and 10.200.221.100 is some non-lwIP
Modbus-TCP client.
In that case, it seems like the Modbus-TCP client retransmits 2 requests
in one TCP segment (note that frame 2 and 3 have a TCP length of 12
while frame 5 has 24).
This is perfectly valid for TCP. It seems like your application cannot
handle 2 requests coming in with one "recv" callback.

This is a common error people do when programming TCP applications: you
get a stream, not packets. You need to handle the depacketization at
application level.

Simon

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: TCP retransmission

ankish
In reply to this post by Nikolas Karakotas-3
Hi nikolas,

Are you able to detect the cause of the issue.
We are also facing similar issue.





--
Sent from: http://lwip.100.n7.nabble.com/lwip-users-f3.html

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users