Issues with netconn callbacks

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Issues with netconn callbacks

Davide Bettio
Hello,

I'm the author of AtomVM (https://github.com/bettio/AtomVM) and we are using lwip on ESP32.

We decided to use netconn API since it provides us asynchronous callbacks.
However I'm experiencing an unexpected behavior with it (when using it as a TCP server):
I get my callback called also for netconns which I didn't accept yet.

So I get this output from my code:

handler not found for: 0x3ffc3bf4
processed all events
found handler for: 0x3ffcd464
tcp_server_handler
going to send a ready message
accepted conn: 0x3ffc3bf4
accepted_socket

Is this an intended behavior?

Also I have further questions about the netconn API:

- Is it possible to check if any byte is already in the buffer ready to be received?
- Is it possible to check if a netconn is still open or it has been closed?

Regards,
Davide Bettio.


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Issues with netconn callbacks

goldsimon@gmx.de
Am 02.12.2019 um 20:39 schrieb Davide Bettio:
> Hello,
>
> I'm the author of AtomVM (https://github.com/bettio/AtomVM) and we are
> using lwip on ESP32.

So you're running the stock lwIP that comes with ESP32? If so, I don't
know how "vanilla" this is and how they keep their copy of lwIP up to
date...

>
> We decided to use netconn API since it provides us asynchronous callbacks.
> However I'm experiencing an unexpected behavior with it (when using it
> as a TCP server):
> I get my callback called also for netconns which I didn't accept yet.

I'm not sure I understand that. Just for you to know: netconn is the
basis of the standard socket implementation in lwIP but otherwise not
may not too be widely used.

Being like that, you *may* get problems when using netconn in a
different way than the socket API uses it internally. Maybe you can
cross-check your code against sockets.c and see if you do anything
different?

>
> So I get this output from my code:
>
> handler not found for: 0x3ffc3bf4
> processed all events
> found handler for: 0x3ffcd464
> tcp_server_handler
> going to send a ready message
> accepted conn: 0x3ffc3bf4
> accepted_socket
>
> Is this an intended behavior?

That's no debug output from lwIP

>
> Also I have further questions about the netconn API:
>
> - Is it possible to check if any byte is already in the buffer ready to
> be received?

Yes, you can use nonblocking receive. Again, see sockets.c if you need
an example.

> - Is it possible to check if a netconn is still open or it has been closed?

Well, you get informed it is closed and you'll get an error trying to
use it...

Regards,
Simon

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Issues with netconn callbacks

Davide Bettio
Hello,

Il giorno lun 2 dic 2019 alle ore 21:26 [hidden email] <[hidden email]> ha scritto:
Am 02.12.2019 um 20:39 schrieb Davide Bettio:
So you're running the stock lwIP that comes with ESP32? If so, I don't
know how "vanilla" this is and how they keep their copy of lwIP up to
date...

Yes, I'm using the ESP32 version.
 
>
> We decided to use netconn API since it provides us asynchronous callbacks.
> However I'm experiencing an unexpected behavior with it (when using it
> as a TCP server):
> I get my callback called also for netconns which I didn't accept yet.

I'm not sure I understand that. Just for you to know: netconn is the
basis of the standard socket implementation in lwIP but otherwise not
may not too be widely used.

Do you suggest me to avoid using netconn? Anyway I would appreciate an API which allows me to:

- Receive asynchronous events (callbacks are ok)
- 0 copy to improve efficiency
- No need for a BSD socket API, I'm writing a virtual machine, so everything is wrapped

netconn looked like the right choice for this kind of needs.
Should I use some kind of hybrid approach? like using sockets and somehow callbacks?
 
Being like that, you *may* get problems when using netconn in a
different way than the socket API uses it internally. Maybe you can
cross-check your code against sockets.c and see if you do anything
different?

I'm using it in this way:

struct netconn *conn = netconn_new_with_proto_and_callback(NETCONN_TCP, 0, socket_callback);

err_t status = netconn_bind(conn, IP_ADDR_ANY, port);
if (UNLIKELY(status != ERR_OK)) {
//TODO
fprintf(stderr, "bind error: %i\n", status);
return;
}

ip_addr_t naddr;
u16_t nport;
status = netconn_getaddr(conn, &naddr, &nport, 1);
if (UNLIKELY(status != ERR_OK)) {
//TODO
fprintf(stderr, "getaddr error: %i\n", status);
return;
}

Anyway I keep getting my callback called for netconns I didn't yet accept.

>
> So I get this output from my code:
>
> handler not found for: 0x3ffc3bf4
> processed all events
> found handler for: 0x3ffcd464
> tcp_server_handler
> going to send a ready message
> accepted conn: 0x3ffc3bf4
> accepted_socket
>
> Is this an intended behavior?

That's no debug output from lwIP

Yes, right, it's just a trace of my application.
 
> - Is it possible to check if any byte is already in the buffer ready to
> be received?

Yes, you can use nonblocking receive. Again, see sockets.c if you need
an example.

> - Is it possible to check if a netconn is still open or it has been closed?

Well, you get informed it is closed and you'll get an error trying to
use it...

Yes right, I was trying to understand to if it was possible to get this kind of information by inspecting netconn struct.

Thank you for your help so far.


Regards,
Davide Bettio.

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Issues with netconn callbacks

Davide Bettio
In reply to this post by goldsimon@gmx.de
Hello,

Il giorno lun 2 dic 2019 alle ore 21:26 [hidden email] <[hidden email]> ha scritto:
So you're running the stock lwIP that comes with ESP32? If so, I don't
know how "vanilla" this is and how they keep their copy of lwIP up to
date...

I checked the source. Anyway it is not vanilla, and moreover it is a quite old version, so recv ignores the nonblocking flag.

 
> - Is it possible to check if any byte is already in the buffer ready to
> be received?

Yes, you can use nonblocking receive. Again, see sockets.c if you need
an example.

As I said before there is no nonblocking receive. Are you aware of any other way to check if data is ready to be received or not?
 
Regards,
Davide Bettio.


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Issues with netconn callbacks

goldsimon@gmx.de
In reply to this post by Davide Bettio
Am 03.12.2019 um 01:05 schrieb Davide Bettio:

> Hello,
>
> Il giorno lun 2 dic 2019 alle ore 21:26 [hidden email]
> <mailto:[hidden email]> <[hidden email] <mailto:[hidden email]>>
> ha scritto:
>
>     Am 02.12.2019 um 20:39 schrieb Davide Bettio:
>     So you're running the stock lwIP that comes with ESP32? If so, I don't
>     know how "vanilla" this is and how they keep their copy of lwIP up to
>     date...
>
>
> Yes, I'm using the ESP32 version.
>
>      >
>      > We decided to use netconn API since it provides us asynchronous
>     callbacks.
>      > However I'm experiencing an unexpected behavior with it (when
>     using it
>      > as a TCP server):
>      > I get my callback called also for netconns which I didn't accept yet.
>
>     I'm not sure I understand that. Just for you to know: netconn is the
>     basis of the standard socket implementation in lwIP but otherwise not
>     may not too be widely used.
>
>
> Do you suggest me to avoid using netconn?

No, not at all! I merely wanted to say that it's not as widely used:
people seeking portability use sockets, most people seeking performance
use the raw callback API. The netconn API *is* in use by everyone using
the socket API.

> Anyway I would appreciate an
> API which allows me to:
>
> - Receive asynchronous events (callbacks are ok)
> - 0 copy to improve efficiency
> - No need for a BSD socket API, I'm writing a virtual machine, so
> everything is wrapped
>
> netconn looked like the right choice for this kind of needs.
> Should I use some kind of hybrid approach? like using sockets and
> somehow callbacks?

No, don't try that, it won't work without modifying the sources.

>
>     Being like that, you *may* get problems when using netconn in a
>     different way than the socket API uses it internally. Maybe you can
>     cross-check your code against sockets.c and see if you do anything
>     different?
>
>
> I'm using it in this way:
>
> struct netconn *conn = netconn_new_with_proto_and_callback(NETCONN_TCP,
> 0, socket_callback);
>
>
> err_t status = netconn_bind(conn, IP_ADDR_ANY, port);
>
> if (UNLIKELY(status != ERR_OK)) {
>
> //TODO
>
> fprintf(stderr, "bind error: %i\n", status);
>
> return;
>
> }
>
>
> ip_addr_t naddr;
>
> u16_t nport;
>
> status = netconn_getaddr(conn, &naddr, &nport, 1);
>
> if (UNLIKELY(status != ERR_OK)) {
>
> //TODO
>
> fprintf(stderr, "getaddr error: %i\n", status);
>
> return;
>
> }
>
> Anyway I keep getting my callback called for netconns I didn't yet accept.

Oh, OK, I think I understand. Have a look at sockets.c:

- in event_callback(), we check if the netconn is already connected to a
socket (conn->callback_arg.socket in current git master). If that is <
0, it's not yet connected. To keep track of "receive events", which we
count only for the socket layer, not for the netconn layer, we use this
socket member to count into the negative. conn->callback_arg.socket is
-1 by default unless initialized to a socket (which is >= 0), so if it
is -2, we had one RCVPLUS event (SENDPLUS cannot happen at that stage).

All this can happen because the stack internally accepted the connection
(SYN-ACK, ACK), so the remote side is free to send data. Internally, the
new netconn is then put on the listener's netconn->acceptmbox, and what
I described above is the situation before an application thread takes
this new netconn out of that acceptmbox (via netconn_accept()).

- lwip_accept() takes a netconn out of the listener's
netconn->acceptmbox and allocates a socket for it. To prevent missing
RCVPLUS events that have already happened, we check for
newconn->callback_arg.socket begin < -1.

And here you see what I mean: all this could well be hidden inside
netconn and the socket API could be a thinner wrapper around netconn,
mainly just wrapping structs and memcpy code. But being like it is, you
might miss some things you know from sockets when using the netconn API.

>
>      >
>      > So I get this output from my code:
>      >
>      > handler not found for: 0x3ffc3bf4
>      > processed all events
>      > found handler for: 0x3ffcd464
>      > tcp_server_handler
>      > going to send a ready message
>      > accepted conn: 0x3ffc3bf4
>      > accepted_socket
>      >
>      > Is this an intended behavior?
>
>     That's no debug output from lwIP
>
>
> Yes, right, it's just a trace of my application.
>
>      > - Is it possible to check if any byte is already in the buffer
>     ready to
>      > be received?
>
>     Yes, you can use nonblocking receive. Again, see sockets.c if you need
>     an example.
>
>      > - Is it possible to check if a netconn is still open or it has
>     been closed?
>
>     Well, you get informed it is closed and you'll get an error trying to
>     use it...
>
>
> Yes right, I was trying to understand to if it was possible to get this
> kind of information by inspecting netconn struct.

No, not really. You have to keep track.

Regards,
Simon

>
> Thank you for your help so far.
>
>
> Regards,
> Davide Bettio.
>
> _______________________________________________
> lwip-users mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/lwip-users
>


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Issues with netconn callbacks

goldsimon@gmx.de
In reply to this post by Davide Bettio
Am 03.12.2019 um 18:27 schrieb Davide Bettio:

> Hello,
>
> Il giorno lun 2 dic 2019 alle ore 21:26 [hidden email]
> <mailto:[hidden email]> <[hidden email] <mailto:[hidden email]>>
> ha scritto:
>
>     So you're running the stock lwIP that comes with ESP32? If so, I don't
>     know how "vanilla" this is and how they keep their copy of lwIP up to
>     date...
>
>
> I checked the source. Anyway it is not vanilla, and moreover it is a
> quite old version, so recv ignores the nonblocking flag.

I know it's not, which is sad. They haven't responded to my offer of
helping them to mainline things they added (or solving them differently
so they don't need to patch our sources). And not needing to patch our
sources, an upgrade would be much easier...

>
>
>      > - Is it possible to check if any byte is already in the buffer
>     ready to
>      > be received?
>
>     Yes, you can use nonblocking receive. Again, see sockets.c if you need
>     an example.
>
> As I said before there is no nonblocking receive. Are you aware of any
> other way to check if data is ready to be received or not?

Well, you have to keep track of RCVPLUS callbacks for a netconn. For
every pbuf/netbuf enqueued into the recvmbox, you get one RCVPLUS call.
That's how the socket API does it with select/recv, so even with
blocking sockets, you won't actually block.

Regards,
Simon

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users