handle slow TCP handling

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

handle slow TCP handling

Pierre Chatelier
Hello,

I have a question regarding the expected behaviour with lwip incoming TCP packets.

I have a my_tcp_recv_cb() callback that handles incoming TCP packets.
Usually, it just ends with
tcp_recved(pcb, p->tot_len);
pbuf_free(p);

However, if sometimes I do not have enough time or memory to handle the packet, I would like to tell "do not ack this one, please send it again later"
According to the doc (http://lwip.wikia.com/wiki/Raw/TCP), I understand that I should :
    -call tcp_recved(pcb, written);//if I was still able to handle "written" bytes, and in that case obviously "written" is less than p->tot_len
    -do *not* call pbuf_free(p);
    -return an error (e.g. ERR_MEM)

"
If there are no errors and the callback function returns ERR_OK, then it is responsible for freeing the pbuf. Otherwise, it must not free the pbuf so that lwIP core code can store it."

But in that case, it just starts to be a mess. Some packets begin to be delivered very slowly (1 byte at a time in pbufs!), and it does not seem to behave as expected.
Meanwhile, the program that sent all the data to the network didn't notice anything, it has just been informed from its TCP API that everything was delivered.

Am I doing wrong ? Did I miss something ?

To be comprehensive, here are some options of my lwip instance :
 PARAMETER LIBRARY_NAME = lwip202
 PARAMETER LIBRARY_VER = 1.1
 PARAMETER PROC_INSTANCE = ps7_cortexa9_0
 PARAMETER ip_frag_max_mtu = 9000
 PARAMETER lwip_dhcp = true
 PARAMETER mem_size = 524288
 PARAMETER memp_n_pbuf = 1024
 PARAMETER memp_n_tcp_seg = 1024
 PARAMETER pbuf_pool_bufsize = 9700
 PARAMETER tcp_ip_rx_checksum_offload = true
 PARAMETER tcp_ip_tx_checksum_offload = true
 PARAMETER tcp_mss = 8060
 PARAMETER tcp_rx_checksum_offload = true
 PARAMETER tcp_snd_buf = 65535
 PARAMETER tcp_tx_checksum_offload = true
 PARAMETER tcp_wnd = 65535


-- 
-------------------------------------------
Pierre Chatelier
[hidden email]
(+33) 03 80 71 16 16
-------------------------------------------

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: handle slow TCP handling

goldsimon@gmx.de
On 27.11.2018 17:16, Pierre Chatelier wrote:
Hello,

I have a question regarding the expected behaviour with lwip incoming TCP packets.

I have a my_tcp_recv_cb() callback that handles incoming TCP packets.
Usually, it just ends with
tcp_recved(pcb, p->tot_len);
pbuf_free(p);

However, if sometimes I do not have enough time or memory to handle the packet, I would like to tell "do not ack this one, please send it again later"
According to the doc (http://lwip.wikia.com/wiki/Raw/TCP), I understand that I should :

I didn' know that one. The wiki is not as well maintained as it should be, so it's always better to read the main documentation at http://www.nongnu.org/lwip/

    -call tcp_recved(pcb, written);//if I was still able to handle "written" bytes, and in that case obviously "written" is less than p->tot_len
    -do *not* call pbuf_free(p);
    -return an error (e.g. ERR_MEM)

No, you cannot combine "tcp_recved" with returning != ERR_OK. Because when returning ERR_MEM, the stack will buffer this pbuf and pass it back to you later. You'll end up with calling tcp_recved for some bytes more than once.


"
If there are no errors and the callback function returns ERR_OK, then it is responsible for freeing the pbuf. Otherwise, it must not free the pbuf so that lwIP core code can store it."

But in that case, it just starts to be a mess. Some packets begin to be delivered very slowly (1 byte at a time in pbufs!), and it does not seem to behave as expected.

Actually, that *is* expected. Returning ERR_MEM can only be the last resort. It destroys some TCP assumptions. You should rather keep the pbuf on a list for yourself and return ERR_OK and process it at some later time, calling 'tcp_recved' after processing it. This will send ACKs immediately (TCP depends on that) but the window will close and will be reopened for the remote host to send more once you called 'tcp_recved'.

If you can't buffer all pbufs, decrease your TCP_WND size.


Simon

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: handle slow TCP handling

Pierre Chatelier
Hi,

Ok, so it seems I misunderstood the doc.
I will try to keep the pbufs in some queue for delayed processing.

Just a last question : 
If I am really slow, I will have to queue many pbufs, and I will exhaust the pool of pbufs for the new incoming TCP packets.
I expect that in that case, the emitting program client will be told that something could not be delivered, and it will have to handle that explicitely. Am I right this case ? Or is the "running out of pbuf" error designed to be handled differently ?




On 27.11.2018 17:16, Pierre Chatelier wrote:
Hello,

I have a question regarding the expected behaviour with lwip incoming TCP packets.

I have a my_tcp_recv_cb() callback that handles incoming TCP packets.
Usually, it just ends with
tcp_recved(pcb, p->tot_len);
pbuf_free(p);

However, if sometimes I do not have enough time or memory to handle the packet, I would like to tell "do not ack this one, please send it again later"
According to the doc (http://lwip.wikia.com/wiki/Raw/TCP), I understand that I should :

I didn' know that one. The wiki is not as well maintained as it should be, so it's always better to read the main documentation at http://www.nongnu.org/lwip/

    -call tcp_recved(pcb, written);//if I was still able to handle "written" bytes, and in that case obviously "written" is less than p->tot_len
    -do *not* call pbuf_free(p);
    -return an error (e.g. ERR_MEM)

No, you cannot combine "tcp_recved" with returning != ERR_OK. Because when returning ERR_MEM, the stack will buffer this pbuf and pass it back to you later. You'll end up with calling tcp_recved for some bytes more than once.


"
If there are no errors and the callback function returns ERR_OK, then it is responsible for freeing the pbuf. Otherwise, it must not free the pbuf so that lwIP core code can store it."

But in that case, it just starts to be a mess. Some packets begin to be delivered very slowly (1 byte at a time in pbufs!), and it does not seem to behave as expected.

Actually, that *is* expected. Returning ERR_MEM can only be the last resort. It destroys some TCP assumptions. You should rather keep the pbuf on a list for yourself and return ERR_OK and process it at some later time, calling 'tcp_recved' after processing it. This will send ACKs immediately (TCP depends on that) but the window will close and will be reopened for the remote host to send more once you called 'tcp_recved'.

If you can't buffer all pbufs, decrease your TCP_WND size.


Simon
_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users