losing input packets and repeated acks

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

losing input packets and repeated acks

Carl D. Blake
I'm having a problem with the input side of TCP.  I am receiving a
fairly large amount of data (approximately 1 megabyte).  Periodically,
(about every 50,000 to 160,000 bytes) the transfer hangs.  The
transmitter is waiting for an ack from my lwip device and keeps retrying
a packet.  Several seconds later my lwip device suddenly responds with
several acks of a packet that has already been acked.  I've seen the
system do this for up to 15 seconds before my lwip device finally acks
the packet that is being retried.  After that the transmission proceeds
normally for another 50,000 to 160,000 bytes and then it happens again.

I've attached my lwipopts.h file and a tcpdump trace.



_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users

lwipopts.h (4K) Download Attachment
mptst (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Jonathan Larmour
Carl D. Blake wrote:
> I'm having a problem with the input side of TCP.  I am receiving a
> fairly large amount of data (approximately 1 megabyte).  Periodically,
> (about every 50,000 to 160,000 bytes) the transfer hangs.  The
> transmitter is waiting for an ack from my lwip device and keeps retrying
> a packet.  Several seconds later my lwip device suddenly responds with
> several acks of a packet that has already been acked.  I've seen the
> system do this for up to 15 seconds before my lwip device finally acks
> the packet that is being retried.  After that the transmission proceeds
> normally for another 50,000 to 160,000 bytes and then it happens again.

I have found it very useful to enable LWIP_STATS in lwipopts.h, and in my
debugger print out the lwip_stats structure, and see which reports errors,
particularly memory errors from mem, memp or pbuf allocations. That
usually points to the culprit.

If you want to print the stats manually in your program you can enable
LWIP_STATS_DISPLAY, #include "lwip/stats.h", and call stats_display();
Assuming your LWIP_PLATFORM_DIAG macro is set in your port to something
appropriate anyway.

Jifl
--
eCosCentric    http://www.eCosCentric.com/    The eCos and RedBoot experts
Company legal info, address and number:   http://www.ecoscentric.com/legal
------["The best things in life aren't things."]------      Opinions==mine


_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Jonathan Larmour
In reply to this post by Carl D. Blake
Carl D. Blake wrote:
>
> I've attached my lwipopts.h file and a tcpdump trace.

I just noticed a few things in your config too...

> /* #define TCP_TMR_INTERVAL 250 */
> #define TCP_TMR_INTERVAL 10

That is very very small. You will probably be too busy processing the TCP
timer to do any real work. The default should be adequate.

> #define PBUF_POLL_SIZE 30

This should be PBUF_POOL_SIZE, not POLL. You will therefore have 16
buffers by default instead.

> #define PBUF_POOL_BUFSIZE 1536

Are you sure you mean 1536 and not 1500?

> #define PBUF_LINK_HLEN 16

Are you sure you mean 16 and not 14?

> #define TCP_MSS 1476

Again I would have expected this to be 1460 (just like your peer in the
trace below). Fortunately for you, your peer won't go above 1460, so you
won't see any problem, but this value still seems wrong, unless there's
something unconventional about your setup.

> #define TCP_WND (16 * 1024)

16K, but you only have 16 pbufs, so it will be easy to run out of pbufs.
You will use 1 pbuf even if you only receive a tiny packet, and that
includes acks. It does dependent somewhat on your data pattern, but it is
usually more efficient to have a larger number of smaller pbufs. It's
usually better to have 3 500-byte pbufs than 1 1500-byte pbuf. Remember
that pbufs can be chained.

> ------------------------------------------------------------------------
>
> 13:53:58.646727 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P 904882:904904(22) ack 1 win 5840 (DF)
> 13:53:58.649988 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: . 904904:906364(1460) ack 1 win 5840 (DF)
> 13:53:58.651482 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: . 906364:907824(1460) ack 1 win 5840 (DF)
> 13:53:58.654904 10.0.1.222.4098 > boromir.tucson.boeckeler.com.10001: . ack 906364 win 11094 (DF)
> 13:53:58.654912 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P 907824:909284(1460) ack 1 win 5840 (DF)
> 13:53:58.654915 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P 909284:909926(642) ack 1 win 5840 (DF)
> 13:53:58.655868 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: . 909926:911386(1460) ack 1 win 5840 (DF)
> 13:53:58.661166 10.0.1.222.4098 > boromir.tucson.boeckeler.com.10001: . ack 906364 win 12576 (DF)

Nevermind the huge delay later, you are getting problems right here. You
will probably find you are out of pbufs. You are advertising a larger
window than you have memory to accept, so the remote size is still
flinging data at you. If you allow out of sequence packets, this may also
explain the large delays, as they will use up pbufs waiting for any
missing data, but there would be no pbufs left to allow the missing data
to come in!

Jifl
--
eCosCentric    http://www.eCosCentric.com/    The eCos and RedBoot experts
Company legal info, address and number:   http://www.ecoscentric.com/legal
------["The best things in life aren't things."]------      Opinions==mine


_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Carl D. Blake
Thank you for your help.  I've listed the things I've tried below.

On Fri, 2007-01-12 at 19:53, Jonathan Larmour wrote:

> Carl D. Blake wrote:
> >
> > I've attached my lwipopts.h file and a tcpdump trace.
>
> I just noticed a few things in your config too...
>
> > /* #define TCP_TMR_INTERVAL 250 */
> > #define TCP_TMR_INTERVAL 10
>
> That is very very small. You will probably be too busy processing the TCP
> timer to do any real work. The default should be adequate.
>
I set it back to 250.  I noticed that I was getting some 250 ms delays
between packets, so I reduced this to try and improve throughput.

> > #define PBUF_POLL_SIZE 30
>
> This should be PBUF_POOL_SIZE, not POLL. You will therefore have 16
> buffers by default instead.

Oops.  I changed the name to PBUF_POOL_SIZE and set it to 90.
>
> > #define PBUF_POOL_BUFSIZE 1536
>
> Are you sure you mean 1536 and not 1500?
>

Per your suggestion below.  I changed this to 500.  I was trying to set
it to the maximum size of my ethernet packet aligned to 8 bytes.

> > #define PBUF_LINK_HLEN 16
>
> Are you sure you mean 16 and not 14?
>

I definitely mean 16 here.  I tried 14, but some combination of lwip,
the GNU C compiler V3.4.4, MEM_ALIGNMENT set to 4, and my processor
(MPC8271) makes the whole thing not work if HLEN is 14.  16 seems to
work.

> > #define TCP_MSS 1476
>
> Again I would have expected this to be 1460 (just like your peer in the
> trace below). Fortunately for you, your peer won't go above 1460, so you
> won't see any problem, but this value still seems wrong, unless there's
> something unconventional about your setup.
>
I changed this to 1460.  I saw 1476 in some lwip example somewhere.

> > #define TCP_WND (16 * 1024)
>
> 16K, but you only have 16 pbufs, so it will be easy to run out of pbufs.
> You will use 1 pbuf even if you only receive a tiny packet, and that
> includes acks. It does dependent somewhat on your data pattern, but it is
> usually more efficient to have a larger number of smaller pbufs. It's
> usually better to have 3 500-byte pbufs than 1 1500-byte pbuf. Remember
> that pbufs can be chained.
>
OK.  I've changed my setup to have 90 pbufs.  Each one is 500 bytes.

> > ------------------------------------------------------------------------
> >
> > 13:53:58.646727 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P 904882:904904(22) ack 1 win 5840 (DF)
> > 13:53:58.649988 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: . 904904:906364(1460) ack 1 win 5840 (DF)
> > 13:53:58.651482 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: . 906364:907824(1460) ack 1 win 5840 (DF)
> > 13:53:58.654904 10.0.1.222.4098 > boromir.tucson.boeckeler.com.10001: . ack 906364 win 11094 (DF)
> > 13:53:58.654912 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P 907824:909284(1460) ack 1 win 5840 (DF)
> > 13:53:58.654915 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P 909284:909926(642) ack 1 win 5840 (DF)
> > 13:53:58.655868 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: . 909926:911386(1460) ack 1 win 5840 (DF)
> > 13:53:58.661166 10.0.1.222.4098 > boromir.tucson.boeckeler.com.10001: . ack 906364 win 12576 (DF)
>
> Nevermind the huge delay later, you are getting problems right here. You
> will probably find you are out of pbufs. You are advertising a larger
> window than you have memory to accept, so the remote size is still
> flinging data at you. If you allow out of sequence packets, this may also
> explain the large delays, as they will use up pbufs waiting for any
> missing data, but there would be no pbufs left to allow the missing data
> to come in!
>
I made the changes you suggested and things are a little better.  I'm
still getting the huge delay periodically, but the multiple acks of the
same packet and the same window size are gone.  I still get multiple
acks of the same packet, but the system seems to be advertising a larger
window size for each ack.  I seem to be getting some corrupt data, but
it's not nearly as bad as it was before.

I enabled LWIP_STATS and printed out the stats before and after the
transfer.  TCP stats is reporting several checksum errors.  PBUF stats
is reporting several errors.  I enabled debug messages for pbufs and I'm
getting the message "pbuf_alloc: Out of pbufs in pool."  This happens
quite a bit during the transfer of 1 Mbyte.  I hadn't expected this to
be a problem since my pbuf memory is 45Kbytes and my TCP window is
16Kbytes.  Any other suggestions?

> Jifl



_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Jonathan Larmour
Carl D. Blake wrote:

> Thank you for your help.  I've listed the things I've tried below.
>
> On Fri, 2007-01-12 at 19:53, Jonathan Larmour wrote:
>> Carl D. Blake wrote:
>>> I've attached my lwipopts.h file and a tcpdump trace.
>> I just noticed a few things in your config too...
>>
>>> /* #define TCP_TMR_INTERVAL 250 */
>>> #define TCP_TMR_INTERVAL 10
>> That is very very small. You will probably be too busy processing the TCP
>> timer to do any real work. The default should be adequate.
>>
> I set it back to 250.  I noticed that I was getting some 250 ms delays
> between packets, so I reduced this to try and improve throughput.

The delays are symptoms rather than cause.

>>> #define PBUF_LINK_HLEN 16
>> Are you sure you mean 16 and not 14?
>>
>
> I definitely mean 16 here.  I tried 14, but some combination of lwip,
> the GNU C compiler V3.4.4, MEM_ALIGNMENT set to 4, and my processor
> (MPC8271) makes the whole thing not work if HLEN is 14.  16 seems to
> work.

Odd. I've used it with GCC 3.4.4, although not PowerPC yet.

> I made the changes you suggested and things are a little better.  I'm
> still getting the huge delay periodically, but the multiple acks of the
> same packet and the same window size are gone.  I still get multiple
> acks of the same packet, but the system seems to be advertising a larger
> window size for each ack.  I seem to be getting some corrupt data, but
> it's not nearly as bad as it was before.

Missing data is one thing, but corrupt data implies to me that there is
something more problematic going on at a lower level, possibly hardware,
more likely your software ethernet device driver.

> I enabled LWIP_STATS and printed out the stats before and after the
> transfer.  TCP stats is reporting several checksum errors.  PBUF stats
> is reporting several errors.  I enabled debug messages for pbufs and I'm
> getting the message "pbuf_alloc: Out of pbufs in pool."  This happens
> quite a bit during the transfer of 1 Mbyte.  I hadn't expected this to
> be a problem since my pbuf memory is 45Kbytes and my TCP window is
> 16Kbytes.  Any other suggestions?

Indeed the remote side should not send more data than the receive window,
although it may use up a few more pbufs with things like acks. I guess you
could try and find out where the pbufs are going - it's possible your
driver might be leaking pbufs. When the stack goes back to quiescent, do
the stats return to the starting state? i.e. record stats, exchange lots of
data for a bit, stop exchanging data, record stats again.

Jifl
--
eCosCentric    http://www.eCosCentric.com/    The eCos and RedBoot experts
Company legal info, address and number:   http://www.ecoscentric.com/legal
------["The best things in life aren't things."]------      Opinions==mine


_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Carl D. Blake
On Mon, 2007-01-15 at 10:48, Jonathan Larmour wrote:

> > I made the changes you suggested and things are a little better.  I'm
> > still getting the huge delay periodically, but the multiple acks of the
> > same packet and the same window size are gone.  I still get multiple
> > acks of the same packet, but the system seems to be advertising a larger
> > window size for each ack.  I seem to be getting some corrupt data, but
> > it's not nearly as bad as it was before.
>
> Missing data is one thing, but corrupt data implies to me that there is
> something more problematic going on at a lower level, possibly hardware,
> more likely your software ethernet device driver.
>
Hmmm.  I'll look at this possibility more closely.

> > I enabled LWIP_STATS and printed out the stats before and after the
> > transfer.  TCP stats is reporting several checksum errors.  PBUF stats
> > is reporting several errors.  I enabled debug messages for pbufs and I'm
> > getting the message "pbuf_alloc: Out of pbufs in pool."  This happens
> > quite a bit during the transfer of 1 Mbyte.  I hadn't expected this to
> > be a problem since my pbuf memory is 45Kbytes and my TCP window is
> > 16Kbytes.  Any other suggestions?
>
> Indeed the remote side should not send more data than the receive window,
> although it may use up a few more pbufs with things like acks. I guess you
> could try and find out where the pbufs are going - it's possible your
> driver might be leaking pbufs. When the stack goes back to quiescent, do
> the stats return to the starting state? i.e. record stats, exchange lots of
> data for a bit, stop exchanging data, record stats again.
>
When the stack is quiescent the stats return to the starting state.  The
PBUF stats show 90 available and 0 used.  I don't think I'm leaking
pbufs.

> Jifl



_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Carl D. Blake
Things are improving, just not perfect yet.  I rewrote the low level
ethernet driver to use pbufs directly.  Doing that seems to have stopped
the pbuf allocation errors.  The maximum used pbufs according to stats
is now 40.  It never uses all of the pbufs anymore.  My huge delay
appears to be gone.

The behavior of the system is now that the first couple times I transmit
1 Mbyte it goes very quickly.  After that the time it takes to transmit
1 Mbyte begins to increase steadily.  Examining the tcpdump shows that
my lwip device that is receiving the data begins to advertise a smaller
and smaller window.  The window never goes back up to 16Kbytes.  I'll
wait a few minutes before starting another transfer and the lwip
device's advertised window starts at the smaller size.  Any ideas?

On Mon, 2007-01-15 at 11:18, Carl D. Blake wrote:

> On Mon, 2007-01-15 at 10:48, Jonathan Larmour wrote:
> > > I made the changes you suggested and things are a little better.  I'm
> > > still getting the huge delay periodically, but the multiple acks of the
> > > same packet and the same window size are gone.  I still get multiple
> > > acks of the same packet, but the system seems to be advertising a larger
> > > window size for each ack.  I seem to be getting some corrupt data, but
> > > it's not nearly as bad as it was before.
> >
> > Missing data is one thing, but corrupt data implies to me that there is
> > something more problematic going on at a lower level, possibly hardware,
> > more likely your software ethernet device driver.
> >
> Hmmm.  I'll look at this possibility more closely.
>
> > > I enabled LWIP_STATS and printed out the stats before and after the
> > > transfer.  TCP stats is reporting several checksum errors.  PBUF stats
> > > is reporting several errors.  I enabled debug messages for pbufs and I'm
> > > getting the message "pbuf_alloc: Out of pbufs in pool."  This happens
> > > quite a bit during the transfer of 1 Mbyte.  I hadn't expected this to
> > > be a problem since my pbuf memory is 45Kbytes and my TCP window is
> > > 16Kbytes.  Any other suggestions?
> >
> > Indeed the remote side should not send more data than the receive window,
> > although it may use up a few more pbufs with things like acks. I guess you
> > could try and find out where the pbufs are going - it's possible your
> > driver might be leaking pbufs. When the stack goes back to quiescent, do
> > the stats return to the starting state? i.e. record stats, exchange lots of
> > data for a bit, stop exchanging data, record stats again.
> >
> When the stack is quiescent the stats return to the starting state.  The
> PBUF stats show 90 available and 0 used.  I don't think I'm leaking
> pbufs.
>
> > Jifl
>
>
>
> _______________________________________________
> lwip-users mailing list
> [hidden email]
> http://lists.nongnu.org/mailman/listinfo/lwip-users
>



_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Carl D. Blake
I haven't been able to determine why my tcp advertised window keeps
decreasing, but I've been investigating why I appear to be receiving
corrupted data.  It seems that I'm not actually getting corrupted data.
What's happening is that the lwip device attempts to transmit a packet
of data, but it can't allocate a tcp seg.  When that happens it throws
away the packet rather than waiting for the number of tcp segs to drop
and trying again.  I've just been using the default of 16 for
MEMP_NUM_TCP_SEG.  Should I increase this?  It seems odd to me that the
system would just throw the packet away.  Is there some way of retrying
the packet?

On Tue, 2007-01-16 at 09:34, Carl D. Blake wrote:

> Things are improving, just not perfect yet.  I rewrote the low level
> ethernet driver to use pbufs directly.  Doing that seems to have stopped
> the pbuf allocation errors.  The maximum used pbufs according to stats
> is now 40.  It never uses all of the pbufs anymore.  My huge delay
> appears to be gone.
>
> The behavior of the system is now that the first couple times I transmit
> 1 Mbyte it goes very quickly.  After that the time it takes to transmit
> 1 Mbyte begins to increase steadily.  Examining the tcpdump shows that
> my lwip device that is receiving the data begins to advertise a smaller
> and smaller window.  The window never goes back up to 16Kbytes.  I'll
> wait a few minutes before starting another transfer and the lwip
> device's advertised window starts at the smaller size.  Any ideas?
>
> On Mon, 2007-01-15 at 11:18, Carl D. Blake wrote:
> > On Mon, 2007-01-15 at 10:48, Jonathan Larmour wrote:
> > > > I made the changes you suggested and things are a little better.  I'm
> > > > still getting the huge delay periodically, but the multiple acks of the
> > > > same packet and the same window size are gone.  I still get multiple
> > > > acks of the same packet, but the system seems to be advertising a larger
> > > > window size for each ack.  I seem to be getting some corrupt data, but
> > > > it's not nearly as bad as it was before.
> > >
> > > Missing data is one thing, but corrupt data implies to me that there is
> > > something more problematic going on at a lower level, possibly hardware,
> > > more likely your software ethernet device driver.
> > >
> > Hmmm.  I'll look at this possibility more closely.
> >
> > > > I enabled LWIP_STATS and printed out the stats before and after the
> > > > transfer.  TCP stats is reporting several checksum errors.  PBUF stats
> > > > is reporting several errors.  I enabled debug messages for pbufs and I'm
> > > > getting the message "pbuf_alloc: Out of pbufs in pool."  This happens
> > > > quite a bit during the transfer of 1 Mbyte.  I hadn't expected this to
> > > > be a problem since my pbuf memory is 45Kbytes and my TCP window is
> > > > 16Kbytes.  Any other suggestions?
> > >
> > > Indeed the remote side should not send more data than the receive window,
> > > although it may use up a few more pbufs with things like acks. I guess you
> > > could try and find out where the pbufs are going - it's possible your
> > > driver might be leaking pbufs. When the stack goes back to quiescent, do
> > > the stats return to the starting state? i.e. record stats, exchange lots of
> > > data for a bit, stop exchanging data, record stats again.
> > >
> > When the stack is quiescent the stats return to the starting state.  The
> > PBUF stats show 90 available and 0 used.  I don't think I'm leaking
> > pbufs.
> >
> > > Jifl
> >
> >
> >
> > _______________________________________________
> > lwip-users mailing list
> > [hidden email]
> > http://lists.nongnu.org/mailman/listinfo/lwip-users
> >
>
>
>
> _______________________________________________
> lwip-users mailing list
> [hidden email]
> http://lists.nongnu.org/mailman/listinfo/lwip-users
>



_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Jonathan Larmour
Carl D. Blake wrote:

> I haven't been able to determine why my tcp advertised window keeps
> decreasing, but I've been investigating why I appear to be receiving
> corrupted data.  It seems that I'm not actually getting corrupted data.
> What's happening is that the lwip device attempts to transmit a packet
> of data, but it can't allocate a tcp seg.  When that happens it throws
> away the packet rather than waiting for the number of tcp segs to drop
> and trying again.  I've just been using the default of 16 for
> MEMP_NUM_TCP_SEG.  Should I increase this?  It seems odd to me that the
> system would just throw the packet away.  Is there some way of retrying
> the packet?

IMHO the correct solution would be to increase the number of SEGs. If you
want to retry the packet, that means putting packets on queues, which means
you have to worry about the packets staying there and not being freed back
to the system; plus queue management, plus some event mechanism to trigger
the retry. This all takes code and RAM resources, and risks causing
different resource failures.

The general philosophy in lwIP is to just drop packets when out of
resources. The alternative approach to queue packets would, if extended to
cover all situations where that could be argued, lead to large memory
footprints.

Out of interest, I've personally made my number of TCP_SEGs match
TCP_SND_QUEUELEN, although that would be overkill for some applications. On
a 32-bit architecture, segs are only 20 bytes each.

Jifl
--
--["No sense being pessimistic, it wouldn't work anyway"]-- Opinions==mine


_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: losing input packets and repeated acks

Carl D. Blake
On Tue, 2007-01-16 at 12:48, Jonathan Larmour wrote:

> Carl D. Blake wrote:
> > I haven't been able to determine why my tcp advertised window keeps
> > decreasing, but I've been investigating why I appear to be receiving
> > corrupted data.  It seems that I'm not actually getting corrupted data.
> > What's happening is that the lwip device attempts to transmit a packet
> > of data, but it can't allocate a tcp seg.  When that happens it throws
> > away the packet rather than waiting for the number of tcp segs to drop
> > and trying again.  I've just been using the default of 16 for
> > MEMP_NUM_TCP_SEG.  Should I increase this?  It seems odd to me that the
> > system would just throw the packet away.  Is there some way of retrying
> > the packet?
>
> IMHO the correct solution would be to increase the number of SEGs. If you
> want to retry the packet, that means putting packets on queues, which means
> you have to worry about the packets staying there and not being freed back
> to the system; plus queue management, plus some event mechanism to trigger
> the retry. This all takes code and RAM resources, and risks causing
> different resource failures.
>
> The general philosophy in lwIP is to just drop packets when out of
> resources. The alternative approach to queue packets would, if extended to
> cover all situations where that could be argued, lead to large memory
> footprints.
>
> Out of interest, I've personally made my number of TCP_SEGs match
> TCP_SND_QUEUELEN, although that would be overkill for some applications. On
> a 32-bit architecture, segs are only 20 bytes each.
>
> Jifl
OK, I've ironed out several problems (most of them mine) and the system
is working much better.  Increasing MEMP_NUM_TCP_SEG to 255 helped with
memory errors (they're gone completely).  I discovered one of my
problems where I was dropping an entire section of my data.  This was
what I thought was data corruption.  I am now able to transmit the data
correctly and consistently.  Now I'm trying to get it to go faster.
Currently on a 10 Mbps network I'm transferring at 1 Mbps data rate.  I
would like this to approach 5 Mbps at least.  In looking at the tcpdump
I notice that periodically the lwip device is waiting 280 to 450 ms
before transmitting the next section of data.  Normally the delay is
approximately 20 to 40 ms, but every now and then the delay jumps up.
The tcp window of the receiving end is fairly large (62780).  The data
has all been acked properly (it's not waiting for an ack from the
receiving end).  It just stops for 280 to 450 ms.  I'm examining my code
to see if I can locate a problem there, but I'm wondering if the lwip
tcp stack might contribute to this for some reason.

Thank you so much for your help.  I'm learning quite a bit.




_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users