Sending a large amount of data using UDP

[Jon Bean]

Hi

I have an application were I need to send around 4k of data using UDP.
What I would like to know is how I go about sending this using LWIP. I
can send small packets no problem. I first allocate a pbuf. Then copy my
data to the pbuf. Then send the data. But sending 4k of data doesnt
work. I realise that the Ethernet has a max payload of 1500 bytes. Is
there a different way to send large amounts of data? Do I need to
configure something and then do it the same way I have previously? I am
using a embedded TIVA micro controller.

Thanks

Jon


--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

[Patrick Klos-2]

On 7/19/2020 6:44 AM, Jon Bean wrote:

> Hi
>
> I have an application were I need to send around 4k of data using UDP.
> What I would like to know is how I go about sending this using LWIP. I
> can send small packets no problem. I first allocate a pbuf. Then copy
> my data to the pbuf. Then send the data. But sending 4k of data doesnt
> work. I realise that the Ethernet has a max payload of 1500 bytes. Is
> there a different way to send large amounts of data? Do I need to
> configure something and then do it the same way I have previously? I
> am using a embedded TIVA micro controller.
>
> Thanks
>
> Jon

Have you enabled IP fragmentation? (IP_FRAG)

Or you could just break up the data yourself, add a little custom
header, and essentially fragment the data yourself.  Depending on what's
in this data, you could add sequence numbers and a way for the peer to
report they missed a chunk?  How often are you sending these blocks of data?

Patrick


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

[Jon Bean]

Hi Patrick

Yes I did try with fragmentation enabled but that didn't seem to work. I
can see in wireshark that there is a packet with IP4 type that mentions
fragmentation , but I don't get any UDP packets.

I also did a test where I just send two packets of 1400, one after
another, but only one get sent. I am wondering if I need to check some
send buffer like in TCP?

If I just allocate a pbuf of 2800 and then send it I dont actually get
any errors, it just doesnt send anything.

The problem is that I cant really do anything with the data in terms of
adding sequence numbers, as I am developing the code for the TIVA for a
pc application that is already defined. I kind of thought that UDP could
send up to 64k. I am a bit surprised I cant just ask LWIP to send a
large amount of data and then it send multiple packets.

Regards

Jon


On 19/07/2020 15:18, Patrick Klos wrote:

> On 7/19/2020 6:44 AM, Jon Bean wrote:
>> Hi
>>
>> I have an application were I need to send around 4k of data using
>> UDP. What I would like to know is how I go about sending this using
>> LWIP. I can send small packets no problem. I first allocate a pbuf.
>> Then copy my data to the pbuf. Then send the data. But sending 4k of
>> data doesnt work. I realise that the Ethernet has a max payload of
>> 1500 bytes. Is there a different way to send large amounts of data?
>> Do I need to configure something and then do it the same way I have
>> previously? I am using a embedded TIVA micro controller.
>>
>> Thanks
>>
>> Jon
>
> Have you enabled IP fragmentation? (IP_FRAG)
>
> Or you could just break up the data yourself, add a little custom
> header, and essentially fragment the data yourself.  Depending on
> what's in this data, you could add sequence numbers and a way for the
> peer to report they missed a chunk?  How often are you sending these
> blocks of data?
>
> Patrick
>
>
> _______________________________________________
> lwip-users mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/lwip-users

--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

[Patrick Klos-2]

On 7/19/2020 10:27 AM, Jon Bean wrote:
> Hi Patrick
>
> Yes I did try with fragmentation enabled but that didn't seem to work.
> I can see in wireshark that there is a packet with IP4 type that
> mentions fragmentation , but I don't get any UDP packets.

Does Wireshark reassemble the fragments into a whole 4K UDP packet? Are
you willing/able to share the Wireshark trace file (.pcap)?

> I also did a test where I just send two packets of 1400, one after
> another, but only one get sent. I am wondering if I need to check some
> send buffer like in TCP?

I've had no problem using UDP reliably on the TIVA with LwIP.  Did you
allocate enough buffers?

> If I just allocate a pbuf of 2800 and then send it I dont actually get
> any errors, it just doesnt send anything.

Unless fragmentation was enabled, that's not going to get very far.

> The problem is that I cant really do anything with the data in terms
> of adding sequence numbers, as I am developing the code for the TIVA
> for a pc application that is already defined. I kind of thought that
> UDP could send up to 64k. I am a bit surprised I cant just ask LWIP to
> send a large amount of data and then it send multiple packets.

So the receiver is already expecting a 4K UDP packet?  That kind of ties
your hands, but I'm (pretty) sure you can get it to work with LwIP on
the TIVA*.

Patrick

* I don't think I've ever actually use IP fragmentation on the TIVA, but
I'm sure it works (and if it doesn't, it can be debugged and fixed -
it's not that complicated)

> Regards
>
> Jon
>
>
> On 19/07/2020 15:18, Patrick Klos wrote:
>> On 7/19/2020 6:44 AM, Jon Bean wrote:
>>> Hi
>>>
>>> I have an application were I need to send around 4k of data using
>>> UDP. What I would like to know is how I go about sending this using
>>> LWIP. I can send small packets no problem. I first allocate a pbuf.
>>> Then copy my data to the pbuf. Then send the data. But sending 4k of
>>> data doesnt work. I realise that the Ethernet has a max payload of
>>> 1500 bytes. Is there a different way to send large amounts of data?
>>> Do I need to configure something and then do it the same way I have
>>> previously? I am using a embedded TIVA micro controller.
>>>
>>> Thanks
>>>
>>> Jon
>>
>> Have you enabled IP fragmentation? (IP_FRAG)
>>
>> Or you could just break up the data yourself, add a little custom
>> header, and essentially fragment the data yourself. Depending on
>> what's in this data, you could add sequence numbers and a way for the
>> peer to report they missed a chunk?  How often are you sending these
>> blocks of data?
>>
>> Patrick
>>
>>
>> _______________________________________________
>> lwip-users mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

[Jon Bean]

Patrick

Looks like its working fine now with the IP fragmentation. I think it
may have been because I didnt have the IP_FRAG_MAX_MTU defined as well
as the IP_FRAG.

Cheers

Jon


On 19/07/2020 15:49, Patrick Klos wrote:

> On 7/19/2020 10:27 AM, Jon Bean wrote:
>> Hi Patrick
>>
>> Yes I did try with fragmentation enabled but that didn't seem to
>> work. I can see in wireshark that there is a packet with IP4 type
>> that mentions fragmentation , but I don't get any UDP packets.
>
> Does Wireshark reassemble the fragments into a whole 4K UDP packet?
> Are you willing/able to share the Wireshark trace file (.pcap)?
>
>> I also did a test where I just send two packets of 1400, one after
>> another, but only one get sent. I am wondering if I need to check
>> some send buffer like in TCP?
>
> I've had no problem using UDP reliably on the TIVA with LwIP.  Did you
> allocate enough buffers?
>
>> If I just allocate a pbuf of 2800 and then send it I dont actually
>> get any errors, it just doesnt send anything.
>
> Unless fragmentation was enabled, that's not going to get very far.
>
>> The problem is that I cant really do anything with the data in terms
>> of adding sequence numbers, as I am developing the code for the TIVA
>> for a pc application that is already defined. I kind of thought that
>> UDP could send up to 64k. I am a bit surprised I cant just ask LWIP
>> to send a large amount of data and then it send multiple packets.
>
> So the receiver is already expecting a 4K UDP packet?  That kind of
> ties your hands, but I'm (pretty) sure you can get it to work with
> LwIP on the TIVA*.
>
> Patrick
>
> * I don't think I've ever actually use IP fragmentation on the TIVA,
> but I'm sure it works (and if it doesn't, it can be debugged and fixed
> - it's not that complicated)
>
>> Regards
>>
>> Jon
>>
>>
>> On 19/07/2020 15:18, Patrick Klos wrote:
>>> On 7/19/2020 6:44 AM, Jon Bean wrote:
>>>> Hi
>>>>
>>>> I have an application were I need to send around 4k of data using
>>>> UDP. What I would like to know is how I go about sending this using
>>>> LWIP. I can send small packets no problem. I first allocate a pbuf.
>>>> Then copy my data to the pbuf. Then send the data. But sending 4k
>>>> of data doesnt work. I realise that the Ethernet has a max payload
>>>> of 1500 bytes. Is there a different way to send large amounts of
>>>> data? Do I need to configure something and then do it the same way
>>>> I have previously? I am using a embedded TIVA micro controller.
>>>>
>>>> Thanks
>>>>
>>>> Jon
>>>
>>> Have you enabled IP fragmentation? (IP_FRAG)
>>>
>>> Or you could just break up the data yourself, add a little custom
>>> header, and essentially fragment the data yourself. Depending on
>>> what's in this data, you could add sequence numbers and a way for
>>> the peer to report they missed a chunk? How often are you sending
>>> these blocks of data?
>>>
>>> Patrick
>>>
>>>
>>> _______________________________________________
>>> lwip-users mailing list
>>> [hidden email]
>>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>>
>
>
> _______________________________________________
> lwip-users mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/lwip-users

--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

[Patrick Klos-2]

On 7/19/2020 10:59 AM, Jon Bean wrote:
> Patrick
>
> Looks like its working fine now with the IP fragmentation. I think it
> may have been because I didnt have the IP_FRAG_MAX_MTU defined as well
> as the IP_FRAG.
>
> Cheers
>
> Jon

Good to hear you worked it out!

Patrick

> On 19/07/2020 15:49, Patrick Klos wrote:
>> On 7/19/2020 10:27 AM, Jon Bean wrote:
>>> Hi Patrick
>>>
>>> Yes I did try with fragmentation enabled but that didn't seem to
>>> work. I can see in wireshark that there is a packet with IP4 type
>>> that mentions fragmentation , but I don't get any UDP packets.
>>
>> Does Wireshark reassemble the fragments into a whole 4K UDP packet?
>> Are you willing/able to share the Wireshark trace file (.pcap)?
>>
>>> I also did a test where I just send two packets of 1400, one after
>>> another, but only one get sent. I am wondering if I need to check
>>> some send buffer like in TCP?
>>
>> I've had no problem using UDP reliably on the TIVA with LwIP. Did you
>> allocate enough buffers?
>>
>>> If I just allocate a pbuf of 2800 and then send it I dont actually
>>> get any errors, it just doesnt send anything.
>>
>> Unless fragmentation was enabled, that's not going to get very far.
>>
>>> The problem is that I cant really do anything with the data in terms
>>> of adding sequence numbers, as I am developing the code for the TIVA
>>> for a pc application that is already defined. I kind of thought that
>>> UDP could send up to 64k. I am a bit surprised I cant just ask LWIP
>>> to send a large amount of data and then it send multiple packets.
>>
>> So the receiver is already expecting a 4K UDP packet?  That kind of
>> ties your hands, but I'm (pretty) sure you can get it to work with
>> LwIP on the TIVA*.
>>
>> Patrick
>>
>> * I don't think I've ever actually use IP fragmentation on the TIVA,
>> but I'm sure it works (and if it doesn't, it can be debugged and
>> fixed - it's not that complicated)
>>
>>> Regards
>>>
>>> Jon
>>>
>>>
>>> On 19/07/2020 15:18, Patrick Klos wrote:
>>>> On 7/19/2020 6:44 AM, Jon Bean wrote:
>>>>> Hi
>>>>>
>>>>> I have an application were I need to send around 4k of data using
>>>>> UDP. What I would like to know is how I go about sending this
>>>>> using LWIP. I can send small packets no problem. I first allocate
>>>>> a pbuf. Then copy my data to the pbuf. Then send the data. But
>>>>> sending 4k of data doesnt work. I realise that the Ethernet has a
>>>>> max payload of 1500 bytes. Is there a different way to send large
>>>>> amounts of data? Do I need to configure something and then do it
>>>>> the same way I have previously? I am using a embedded TIVA micro
>>>>> controller.
>>>>>
>>>>> Thanks
>>>>>
>>>>> Jon
>>>>
>>>> Have you enabled IP fragmentation? (IP_FRAG)
>>>>
>>>> Or you could just break up the data yourself, add a little custom
>>>> header, and essentially fragment the data yourself. Depending on
>>>> what's in this data, you could add sequence numbers and a way for
>>>> the peer to report they missed a chunk? How often are you sending
>>>> these blocks of data?
>>>>
>>>> Patrick
>>>>
>>>>
>>>> _______________________________________________
>>>> lwip-users mailing list
>>>> [hidden email]
>>>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>>>
>>
>>
>> _______________________________________________
>> lwip-users mailing list
>> [hidden email]
>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users