Post messages with the httpd.c example

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Post messages with the httpd.c example

Trampas Stern
I was trying to use the httpd.c example which does some POST data parsing.  From the httpd.h file it appears that httpd_post_begin() function is called with the first packet of data.  However in reality it appears in use that it is usually called with just the POST header.  
Since I assume the POST header could span multiple packets I assumed that the  httpd_post_begin() may not have the complete header but might just be first packet in the header.  
Therefore in my implementation of  httpd_post_begin() and httpd_post_receive_data() I was scanning for the double-CRLF to determine the end of the header and start of data. 
What I found is that the "\r\n\r\n" was actually "\0\n\r\n".  Looking through the http_post_request() code it appears that the code is putting the '\0' in the data:

/* trim http header */
*crlfcrlf = 0;
err = httpd_post_begin(hs, uri, hdr_start_after_uri,
hdr_data_len, content_len, http_uri_buf,
LWIP_HTTPD_URI_BUF_LEN, &post_auto_wnd);

Is this an error?   Specifically if the  httpd_post_begin() is not guaranteed to be sent with the full header for the POST then on the users application side we should be managing determining where the header ends and start of data is, and thus the httpd.c code should not be changing the data within the packet. 

Thanks
Trampas



_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Post messages with the httpd.c example

goldsimon@gmx.de


Trampas Stern wrote:

>I was trying to use the httpd.c example which does some POST data
>parsing.
>From the httpd.h file it appears that httpd_post_begin() function is
>called
>with the first packet of data.  However in reality it appears in use
>that
>it is usually called with just the POST header.
>Since I assume the POST header could span multiple packets I assumed
>that
>the  httpd_post_begin() may not have the complete header but might just
>be
>first packet in the header.
>Therefore in my implementation of  httpd_post_begin()
>and httpd_post_receive_data() I was scanning for the double-CRLF to
>determine the end of the header and start of data.
>What I found is that the "\r\n\r\n" was actually "\0\n\r\n".  Looking
>through the http_post_request() code it appears that the code is
>putting
>the '\0' in the data:
>
>/* trim http header */
>* *crlfcrlf = 0;*
>err = httpd_post_begin(hs, uri, hdr_start_after_uri,
>hdr_data_len, content_len, http_uri_buf,
>LWIP_HTTPD_URI_BUF_LEN, &post_auto_wnd);
>
>Is this an error?   Specifically if the  httpd_post_begin() is not
>guaranteed to be sent with the full header for the POST then on the
>users
>application side we should be managing determining where the header
>ends
>and start of data is, and thus the httpd.c code should not be changing
>the
>data within the packet.

From the top of my head, the code buffers incoming pbufs until the complete http header is received. It then trims it to ensure pbuf strlen of the header is the header length.

I'd have to look at it again to be sure though...

Regards,
Simon

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Post messages with the httpd.c example

Trampas Stern
What happens if the MTU size is less than the header size?  Specifically, does the httpd.c code cache the packets and fill a buffer for header until the complete header has been received before calling http_post_begin()?  

From looking at the data transfer it appears that the null terminator is added, but then the length sent to the  http_post_begin() includes the null terminator and three bytes after.  Hence it appears that the null terminator was added to treat the header as a c-string but data is passed with a length like a binary block buffer. 

Tramps

On Tue, Jan 21, 2020 at 4:06 PM goldsimon <[hidden email]> wrote:


Trampas Stern wrote:
>I was trying to use the httpd.c example which does some POST data
>parsing.
>From the httpd.h file it appears that httpd_post_begin() function is
>called
>with the first packet of data.  However in reality it appears in use
>that
>it is usually called with just the POST header.
>Since I assume the POST header could span multiple packets I assumed
>that
>the  httpd_post_begin() may not have the complete header but might just
>be
>first packet in the header.
>Therefore in my implementation of  httpd_post_begin()
>and httpd_post_receive_data() I was scanning for the double-CRLF to
>determine the end of the header and start of data.
>What I found is that the "\r\n\r\n" was actually "\0\n\r\n".  Looking
>through the http_post_request() code it appears that the code is
>putting
>the '\0' in the data:
>
>/* trim http header */
>* *crlfcrlf = 0;*
>err = httpd_post_begin(hs, uri, hdr_start_after_uri,
>hdr_data_len, content_len, http_uri_buf,
>LWIP_HTTPD_URI_BUF_LEN, &post_auto_wnd);
>
>Is this an error?   Specifically if the  httpd_post_begin() is not
>guaranteed to be sent with the full header for the POST then on the
>users
>application side we should be managing determining where the header
>ends
>and start of data is, and thus the httpd.c code should not be changing
>the
>data within the packet.

From the top of my head, the code buffers incoming pbufs until the complete http header is received. It then trims it to ensure pbuf strlen of the header is the header length.

I'd have to look at it again to be sure though...

Regards,
Simon

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Post messages with the httpd.c example

vysocan
My observation is that httpd_post_receive_data is called for each received
packet until content_len for this connection is reached. Then the
httpd_post_finished is called for application to finalize.



--
Sent from: http://lwip.100.n7.nabble.com/lwip-users-f3.html

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Post messages with the httpd.c example

Trampas Stern
The post parsing in the http code has couple problems.  Most of the time it will not cause issues.

First it assume the first packet includes the entire POST header.  With small MTU size this might not happen. 
Second it adds a null terminator at the end of the POST header before calling httpd_post_begin().  This is a minor issue and might be considered a design intent. 

Trampas



On Tue, Feb 4, 2020 at 5:49 AM vysocan <[hidden email]> wrote:
My observation is that httpd_post_receive_data is called for each received
packet until content_len for this connection is reached. Then the
httpd_post_finished is called for application to finalize.



--
Sent from: http://lwip.100.n7.nabble.com/lwip-users-f3.html

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Post messages with the httpd.c example

vysocan
Hello,

small MTU  can be overridden  by TCP_MSS I believe. The default 536 is very small for POST indeed.

Adam

út 4. 2. 2020 v 13:41 odesílatel Trampas Stern <[hidden email]> napsal:
The post parsing in the http code has couple problems.  Most of the time it will not cause issues.

First it assume the first packet includes the entire POST header.  With small MTU size this might not happen. 
Second it adds a null terminator at the end of the POST header before calling httpd_post_begin().  This is a minor issue and might be considered a design intent. 

Trampas



On Tue, Feb 4, 2020 at 5:49 AM vysocan <[hidden email]> wrote:
My observation is that httpd_post_receive_data is called for each received
packet until content_len for this connection is reached. Then the
httpd_post_finished is called for application to finalize.



--
Sent from: http://lwip.100.n7.nabble.com/lwip-users-f3.html

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users


--
731435556
Adam Baron

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users