Allocating and deallocating an mbox in different threads

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Allocating and deallocating an mbox in different threads

Andrew Lentvorski
I have been trying to sort out which functions get called in which
threads when using the sockets API.  Everything is pretty clean except
for the following code in api_lib.c:


     /* If we are closed, we indicate that we no longer wish to receive
        data by setting conn->recvmbox to SYS_MBOX_NULL. */
     if (p == NULL) {
       memp_free(MEMP_NETBUF, buf);
       sys_mbox_free(conn->recvmbox);
       conn->recvmbox = SYS_MBOX_NULL;
       return NULL;
     }

Now, I understand about allocating pbufs and messages in one thread and
deallocating them in another.  That's okay, it doesn't break the
abstraction since those only go one way and all of the information is
explicitly encapsulated in the data.

In this instance, though, the freeing of the mbox is being used as a
reverse message between the API layers.  That's uncool and makes all
kinds of assumptions about the implementation of mbox.

Is there any way to change this to an actual explicit reverse message
and let the original thread do the deallocation?

I don't yet know enough about the API abstractions between the RAW API,
the netconn API, and the socket API to feel that I can generate an
intelligent patch.  Does anybody have a suggestion?

Thanks,
-a



_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Allocating and deallocating an mbox in different threads

Kieran Mansley
On Wed, 2007-04-18 at 03:19 -0700, Andrew Lentvorski wrote:

> In this instance, though, the freeing of the mbox is being used as a
> reverse message between the API layers.  That's uncool and makes all
> kinds of assumptions about the implementation of mbox.
>
> Is there any way to change this to an actual explicit reverse message
> and let the original thread do the deallocation?

Can you give a bit more detail about what problem this is causing?

Kieran



_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Allocating and deallocating an mbox in different threads

Andrew Lentvorski
Kieran Mansley wrote:

> On Wed, 2007-04-18 at 03:19 -0700, Andrew Lentvorski wrote:
>
>> In this instance, though, the freeing of the mbox is being used as a
>> reverse message between the API layers.  That's uncool and makes all
>> kinds of assumptions about the implementation of mbox.
>>
>> Is there any way to change this to an actual explicit reverse message
>> and let the original thread do the deallocation?
>
> Can you give a bit more detail about what problem this is causing?

The problem this is causing is that I have two mbox types of which only
one is "memory-based".

I made a bunch of modifications to separate the two types in the code.
Amazingly, this is a pretty straightforward task, if somewhat tedious.

mbox type A is for intrathread communication.  No problems--this is the
mbox everybody is used to.

mbox type B is for interthread communication.  This is an actual
hardware FIFO between processors.  It is *not* memory-based.

The problem is that deallocation NULL.  That's an implicit message and I
don't get the ability to send implicit messages through a hardware FIFO.

I only get two threads.  Thread 1 is an ARM7, thread 2 is an ARM9.

I can't "just send a deallocation message" as I am already in a polling
loop on the ARM7 inside tcpip_thread().  There is nothing to receive a
"deallocated mbox" message, and I'm at the wrong abstraction level
anyhow for such a low-level message.  I can't abstract the FIFO into
another thread as I don't have another thread to use on both sides.
And, even if I *could* deallocate, the runtimes on both sides are
completely independent so a free() would just throw a segfault or hang
anyway (probably just hang, no memory protection and lots of caching
collisions).

So, my code trucks along and, at best, starts leaking allocated mboxes
on one side.  At worst, it hangs (presumably when it runs out of mboxes).

Basically, Thread 2 needs to send a "connection closed" message at a
slightly higher level of abstraction which "Thread 1" would receive and
then handle its own low-level cleanup.

I don't quite know what the implications of doing this via messages
would be.  That mbox stuff seems to have some strong conditions around
it for when and how it closes in order to avoid race conditions.

-a


_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Allocating and deallocating an mbox in different threads

Kieran Mansley
On Wed, 2007-04-18 at 04:04 -0700, Andrew Lentvorski wrote:

> Kieran Mansley wrote:
> > On Wed, 2007-04-18 at 03:19 -0700, Andrew Lentvorski wrote:
> >
> >> In this instance, though, the freeing of the mbox is being used as a
> >> reverse message between the API layers.  That's uncool and makes all
> >> kinds of assumptions about the implementation of mbox.
> >>
> >> Is there any way to change this to an actual explicit reverse message
> >> and let the original thread do the deallocation?
> >
> > Can you give a bit more detail about what problem this is causing?
>
> The problem this is causing is that I have two mbox types of which only
> one is "memory-based".

So if I understand correctly I think this is specific to the
modifications that you've made?  I can't see a functional problem with
changing this as you suggest to post a message that would then result in
the freeing of the mbox, but I haven't looked at it in detail.  I think
though that as it is specific to your modifications, it's not
appropriate to make this change in the general code base.

Kieran



_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: Allocating and deallocating an mbox in different threads

Andrew Lentvorski
Kieran Mansley wrote:
> So if I understand correctly I think this is specific to the
> modifications that you've made?  I can't see a functional problem with
> changing this as you suggest to post a message that would then result in
> the freeing of the mbox, but I haven't looked at it in detail.  I think
> though that as it is specific to your modifications, it's not
> appropriate to make this change in the general code base.

It would be a problem for anyone who uses an mbox implementation in
sys_arch.c which is not memory/pointer based.  The issue is that the
NULL is effectively a stealth shutdown message.  It is *not* specific to
the fact that I am using 2 threads, multiple mbox-types, etc.

Now, I'm certainly willing to concede that the set of people not using
memory/pointer-based mbox's may consist of exactly *1* person.  ;)

But using that NULL as a shutdown message definitely breaks the
sys_arch.c abstractions that you folks spent so much time setting up.

However, if it's just not a concern, then that's cool, too.  I'll just
hack something in; it's probably easier for me anyhow.  If I don't have
to worry about feeding it back into the main codebase, then I can just
create a quick hack rather than work out all the race conditions to
create a clean patch.

-a


_______________________________________________
lwip-users mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/lwip-users