Using tcp_write() outside tcp_recv callback

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Using tcp_write() outside tcp_recv callback

Jamie
Hi, I'm using lwIP 1.4.1 on a SAME70 with NO_SYS = 1. I started with the tcpech_raw application (a clone of which is linked here for reference), which worked exactly as intended on the SAME70 with the SAME70 port. I've now started to modify the tcpecho_raw application to suit my use case, but am running into a few issues. Before I get into the issues, I think it prudent to detail my use case, my lwipopts and approach.

Use Case

  • lwIP is being used for a direct connection between the SAME70 and another processor on the same board. There will only ever be one connection, using static IP addresses.
  • lwIP is used primarily for TX out of the SAME70. RX is used for configuration updates/changes.
  • The SAME70 receives data from other interfaces, and following some processing, needs to TX the data to the other processor. This is a time-sensitive procedure; the delay between the SAME70 receiving and processing the data, to the data being on the wire can't be more than ~10ms.
  • Each discrete packet of data is less than 50 bytes.
  • lwipopts.h

  • NO_SYS = 1
  • MEM_ALIGNMENT = 4
  • MEM_SIZE = 4 * 1024
  • MEMP_NUM_TCP_PCB = 1
  • LWIP_DHCP = 0
  • TCP_MSS = 1460
  • TCP_WND = (2 * TCP_MSS)
  • TCP_SND_BUF = (2 * TCP_MSS)
  • Approach

  • Maintain the same echo_init function.
  • Use a global struct tcp_pcb and assign it during the accept callback.
  • Use similar tcp_recv and tcp_err functions.
  • I'm not using the tcp_poll function for the time being.
  • Where I need some guidance

    1. In the example echo_send function, after tcp_write(), the current pbuf is freed via a do-while call to pbuf_free(ptr), where struct pbuf *ptr = es->p. I'm not using a send function; I'm looking to use tcp_write() and tcp_output() directly from elsewhere in the application, using the global tcp_pcb assigned on accept callback. How can I free the pbuf following tcp_output? The pbuf_free() function requires a pbuf input, which I don't have when calling tcp_write() from outside a callback. Do I need to manually create and allocate a pbuf, use the pbuf's payload and len parameters in the tcp_write call, and then use pcb_free? At the moment I'm able to send a couple of packets before receiving the "tcp_write: could not allocate memory for pbuf copy size ..." error; I assume this is because the pbuf is not deallocated once used (i.e. after tcp_write and tcp_output).
    2. What should my TCP_OVERSIZE value be?
    3. Do my lwipopts.h defines look reasonable?
    4. Is the approach of sending small packets one at a time feasible? I understand that there is considerable overhead associated with this approach. Depending on my testing (once I've got this approach working), I may look to concatenate multiple packets in a buffer and reduce the number of transmissions, as I should be able to do so within the allowed ~10ms time.
    Any guidance on the above 4 questions would be greatly appreciated! Thanks

    Sent from the lwip-users mailing list archive at Nabble.com.

    _______________________________________________
    lwip-users mailing list
    [hidden email]
    https://lists.nongnu.org/mailman/listinfo/lwip-users
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Using tcp_write() outside tcp_recv callback

    Jamie
    Just following up on this, as I haven't had time to continue working on it
    until today. To provide more context, I'll provide some code snippets and
    commentary around the code. I don't imagine I'm too far off getting it
    working, as I've not changed too much of the echo example source.

    *My init function:*
    init_PCB = tcp_new();

    if (init_PCB != NULL) {

        err_t err;

        err = tcp_bind(init_PCB, IP_ADDR_ANY, 9999);

        if (err == ERR_OK) {
       
            init_PCB = tcp_listen(init_PCB);

            tcp_accept(init_PCB, tcp_accept_callback);

        } else {

            memp_free(MEMP_TCP_PCB, init_PCB);

        }

    }

    My accept callback: (void *arg, struct tcp_pcb *tempPCB, err_t err)
    err_t ret_err;

    struct tcp_package *tcpPackage;

    global_PCB = tempPCB;

    tcp_nagle_disable(global_PCB);

    LWIP_UNUSED_ARG(arg);
    LWIP_UNUSED_ARG(err);

    tcp_setprio(global_PCB, TCP_PRIO_MIN);

    tcpPackage = (struct tcp_package *)mem_malloc(sizeof(struct tcp_package));

    if (tcpPackage != NULL) {

        tcpPackage->state = TCP_STATE_ACCEPTED;
        tcpPackage->pcb = global_PCB;
        tcpPackage->retries = 0;
        tcpPackage->p = NULL;

        tcp_arg(global_PCB, tcpPackage);
        tcp_recv(global_PCB, tcp_recv_callback);
        tcp_err(global_PCB, tcp_err_callback);

        ret_err = ERR_OK;

    } else {

        ret_err = ERR_MEM;

    }

    I'll leave the tcp_recv_callback function for the time being, as I'm
    focusing on TX at the moment, and the RX side seems to work without issue.

    In terms of sending data, I have an interrupt callback function, where some
    data is processed (not shown below for simplicity). I package an unsigned
    char buffer with data from two other buffers, separated by a ';' character.
    Code shown below:

    unsigned char bufferMain[32];
    char buffer1[10];
    char buffer2[10];

    //Some processing of data; buffer1 and buffer2 are populated

    snprintf(bufferMain, sizeof(bufferMain), "%s%s%s", buffer1, ";", buffer2);

    //Register the sent callback
    tcp_sent(global_PCB, tcp_sent_callback);

    err_t err;
    err = tcp_write(global_PCB, bufferMain, sizeof(bufferMain), 1);
    err = tcp_output(global_PCB);

    As mentioned in my original post, I have trace points throughout the code,
    including the 'err' return values for tcp_write() and tcp_output() above.
    When I run the code, I get 2-3 ERR_OK values from tcp_write, and data is
    received by the remote CPU, but err then becomes -1, and (through another
    trace point) I determine the error to be:

    "tcp_write : could not allocate memory for pbuf copy size 32"

    Here is my tcp_sent_callback: (void *arg, struct tcp_pcb *tpcb, u16_t len)
    struct tcp_package *tcpPackage;

    struct pbuf *ptr;

    LWIP_UNUSED_ARGE(len);

    tcpPackage = (struct tcp_package *)arg;

    ptr = tcpPackage->p;

    tcpPackage->retries = 0;

    if (tcpPackage->p != NULL) {

        tcp_sent(tpcb, tcp_sent_callback);

        //I don't use the tcp_send function, so don't currently use this
    functionality
        //In theory I shouldn't need this functionality, as each tcp_write() is
    for a single pbuf
        //tcp_send)tpcb, tcpPackage);

    } else {

        if (tcpPackage->state == TCP_STATE_CLOSING) {
       
            tcpClose(tpcb, tcpPackage);

        }

        uint8_t freePBUF;

        do {

            freePBUF = pbuf_free(ptr);

        } while (freePBUF == 0);

    }

    return ERR_OK;

    Is there anything obvious that I'm missing or have done incorrectly in the
    above? In my mind, I've registered the sent callback, so after tcp_write()
    and tcp_output(), the do-while pbuf_free() should deallocate memory, but it
    looks like that may not be the case, given the "could not allocate memory
    for pbuf copy size" error.

    Any guidance would be greatly appreciated!



    --
    Sent from: http://lwip.100.n7.nabble.com/lwip-users-f3.html

    _______________________________________________
    lwip-users mailing list
    [hidden email]
    https://lists.nongnu.org/mailman/listinfo/lwip-users
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Using tcp_write() outside tcp_recv callback

    goldsimon@gmx.de
    Am 08.04.2020 um 00:08 schrieb Jamie:
    > Just following up on this, as I haven't had time to continue working on it
    > until today. To provide more context, I'll provide some code snippets and
    > commentary around the code. I don't imagine I'm too far off getting it
    > working, as I've not changed too much of the echo example source.

    I'm not sure what you're getting at. This looks a lot different to the
    excho example to me...

    Regards,
    Simon

    >
    > *My init function:*
    > init_PCB = tcp_new();
    >
    > if (init_PCB != NULL) {
    >
    >     err_t err;
    >
    >     err = tcp_bind(init_PCB, IP_ADDR_ANY, 9999);
    >
    >     if (err == ERR_OK) {
    >
    >         init_PCB = tcp_listen(init_PCB);
    >
    >         tcp_accept(init_PCB, tcp_accept_callback);
    >
    >     } else {
    >
    >         memp_free(MEMP_TCP_PCB, init_PCB);
    >
    >     }
    >
    > }
    >
    > My accept callback: (void *arg, struct tcp_pcb *tempPCB, err_t err)
    > err_t ret_err;
    >
    > struct tcp_package *tcpPackage;
    >
    > global_PCB = tempPCB;
    >
    > tcp_nagle_disable(global_PCB);
    >
    > LWIP_UNUSED_ARG(arg);
    > LWIP_UNUSED_ARG(err);
    >
    > tcp_setprio(global_PCB, TCP_PRIO_MIN);
    >
    > tcpPackage = (struct tcp_package *)mem_malloc(sizeof(struct tcp_package));
    >
    > if (tcpPackage != NULL) {
    >
    >     tcpPackage->state = TCP_STATE_ACCEPTED;
    >     tcpPackage->pcb = global_PCB;
    >     tcpPackage->retries = 0;
    >     tcpPackage->p = NULL;
    >
    >     tcp_arg(global_PCB, tcpPackage);
    >     tcp_recv(global_PCB, tcp_recv_callback);
    >     tcp_err(global_PCB, tcp_err_callback);
    >
    >     ret_err = ERR_OK;
    >
    > } else {
    >
    >     ret_err = ERR_MEM;
    >
    > }
    >
    > I'll leave the tcp_recv_callback function for the time being, as I'm
    > focusing on TX at the moment, and the RX side seems to work without issue.
    >
    > In terms of sending data, I have an interrupt callback function, where some
    > data is processed (not shown below for simplicity). I package an unsigned
    > char buffer with data from two other buffers, separated by a ';' character.
    > Code shown below:
    >
    > unsigned char bufferMain[32];
    > char buffer1[10];
    > char buffer2[10];
    >
    > //Some processing of data; buffer1 and buffer2 are populated
    >
    > snprintf(bufferMain, sizeof(bufferMain), "%s%s%s", buffer1, ";", buffer2);
    >
    > //Register the sent callback
    > tcp_sent(global_PCB, tcp_sent_callback);
    >
    > err_t err;
    > err = tcp_write(global_PCB, bufferMain, sizeof(bufferMain), 1);
    > err = tcp_output(global_PCB);
    >
    > As mentioned in my original post, I have trace points throughout the code,
    > including the 'err' return values for tcp_write() and tcp_output() above.
    > When I run the code, I get 2-3 ERR_OK values from tcp_write, and data is
    > received by the remote CPU, but err then becomes -1, and (through another
    > trace point) I determine the error to be:
    >
    > "tcp_write : could not allocate memory for pbuf copy size 32"
    >
    > Here is my tcp_sent_callback: (void *arg, struct tcp_pcb *tpcb, u16_t len)
    > struct tcp_package *tcpPackage;
    >
    > struct pbuf *ptr;
    >
    > LWIP_UNUSED_ARGE(len);
    >
    > tcpPackage = (struct tcp_package *)arg;
    >
    > ptr = tcpPackage->p;
    >
    > tcpPackage->retries = 0;
    >
    > if (tcpPackage->p != NULL) {
    >
    >     tcp_sent(tpcb, tcp_sent_callback);
    >
    >     //I don't use the tcp_send function, so don't currently use this
    > functionality
    >     //In theory I shouldn't need this functionality, as each tcp_write() is
    > for a single pbuf
    >     //tcp_send)tpcb, tcpPackage);
    >
    > } else {
    >
    >     if (tcpPackage->state == TCP_STATE_CLOSING) {
    >
    >         tcpClose(tpcb, tcpPackage);
    >
    >     }
    >
    >     uint8_t freePBUF;
    >
    >     do {
    >
    >         freePBUF = pbuf_free(ptr);
    >
    >     } while (freePBUF == 0);
    >
    > }
    >
    > return ERR_OK;
    >
    > Is there anything obvious that I'm missing or have done incorrectly in the
    > above? In my mind, I've registered the sent callback, so after tcp_write()
    > and tcp_output(), the do-while pbuf_free() should deallocate memory, but it
    > looks like that may not be the case, given the "could not allocate memory
    > for pbuf copy size" error.
    >
    > Any guidance would be greatly appreciated!
    >
    >
    >
    > --
    > Sent from: http://lwip.100.n7.nabble.com/lwip-users-f3.html
    >
    > _______________________________________________
    > lwip-users mailing list
    > [hidden email]
    > https://lists.nongnu.org/mailman/listinfo/lwip-users
    >


    _______________________________________________
    lwip-users mailing list
    [hidden email]
    https://lists.nongnu.org/mailman/listinfo/lwip-users
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Using tcp_write() outside tcp_recv callback

    Jamie
    Hi Simon,

    Thanks for responding. You raise a fair point; I've been through so many
    iterations of changes to the original echo example now it probably doesn't
    carry much resemblance.

    To simplify the issue: I want to send single pbuf packets via tcp_write(),
    and make sure the pbuf is subsequently deallocated. At the moment, I'm able
    to send a handful of packets via tcp_write(), but I always end up with a
    ERR_MEM eventually. I've increased MEM_SIZE to values much larger than
    should be required for the size of my pbuf packet, but it will eventually
    (we're talking seconds here, not minutes or hours) produce the "tcp_write :
    could not allocate memory for pbuf copy size" error.

    The tcp_write() function allocates a pbuf, which should then be deallocated
    in the sent callback? I've tried to mimic the echo sent callback free_pbuf()
    component, the ERR_MEM persists whether I use the sent callback or not.

    What am I missing here?



    --
    Sent from: http://lwip.100.n7.nabble.com/lwip-users-f3.html

    _______________________________________________
    lwip-users mailing list
    [hidden email]
    https://lists.nongnu.org/mailman/listinfo/lwip-users
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Using tcp_write() outside tcp_recv callback

    Jamie
    Morning/evening all,

    I'm following up on this thread, as I'm still at a loss as to how to get this working. I've spent the last week going back and forth on this (among other things), and feel like I have a better understanding of the general process now. I'm still facing the same issue, whereby data is successfully sent out and received by the remote CPU (and the data verified), but transmission always stops soon after commencing the lwIP function. The most frustrating part, is the time elapsed or the amount of data sent, before the transmission stops seems completely random, and doesn't seem affected at all by the lwipopts options. On the other side of the transmission (the remote CPU), I simply tally the amount (measured in bytes) of data received before the transmission stops. I've done an extensive range of testing, whereby I change MEM_SIZE, TCP_SND_QUEUELEN, MEMP_NUM_TCP_SEG, MEMP_NUM_PBUF, etc., and the amount of bytes received before transmission stops seems completely random. I.e. when I ran the program 5 times, with the amount of data and rate of calling tcp_write() remaining constant for each test, I received the following bytes before transmission stopped:

  • 4864
  • 14
  • 830
  • 1943
  • 92
  • I received similar values for a range of different lwipopts as mentioned above. I don't have access to a serial output on this embedded system, so I'm unable to use the lwIP debugging features, nor am I able to capture a wireshark reading. I've attached a .c file showing my function which initiates the sending of data and the tcpSent and tcpSend function. I've removed any irrelevant code, and haven't include my tcpInit(), tcpAccept(), etc. functions, but am happy to add them if it's deemed relevant; I'd greatly appreciate any input :) I'm more than happy to continue 'tuning' the lwipopts values, but I'd really like some verification that the tcp_write(), tcpSent() and tcpSend() processes I've implemented are logical/correct, as I've gone back and forth between so many examples, mailing list posts, tutorials, samples, etc. over the last week or so I don't really know where I stand anymore. I'm almost certain that they are in fact incorrect. It's important to confirm my use case as well, as I think it's probably not a common one. The someFunction() in the attached .c file is called very frequently (potentially >1000 times per second), as it's an IRQ for a range of hardware inputs. The IRQ simply captures a timestamp and packages it with a couple of other characters into a string, and then sends the buffer via lwIP. As such, each buffer being sent with tcp_write() is small (<20 bytes), but there are a lot of them.

    lwIP.c

    Thanks!

    Sent from the lwip-users mailing list archive at Nabble.com.

    _______________________________________________
    lwip-users mailing list
    [hidden email]
    https://lists.nongnu.org/mailman/listinfo/lwip-users
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Using tcp_write() outside tcp_recv callback

    Jamie
    I'm making some progress. I've got my code running on an evaluation kit of my MCU, connected to a PC, so I now have proper serial output for debugging. The main issue I've found so far, is that at some point, lwIP stops sending data out on the wire, causing the snd_buf() availability to gradually/rapidly decrement until it hits 0, causing a 'tcp_write: too much data' error. I've included a screenshot of a plot I've made in Excel, which plots snd_buf() over the number of tcp_write() calls. As you can see, normal behaviour is exhibited (snd_buf() decrements as data is added to the queue, then the data is sent out on the wire, and the buffer is cleared) up until a certain point. At this point, data is no longer being received by the attached computer, and the snd_buf() grows larger and larger until it's full, causing the tcp_write: too much data error. Does anyone have any idea as to why this might be the case? Bear in mind that the rate of calling tcp_write() is fixed, as is the data being sent (it's a sequence of chars making up 20 bytes, which is dynamically allocated in the function which calls tcp_write()).

    Sent from the lwip-users mailing list archive at Nabble.com.

    _______________________________________________
    lwip-users mailing list
    [hidden email]
    https://lists.nongnu.org/mailman/listinfo/lwip-users