IP forwarding to/from PPP working for one netif, but not another

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

IP forwarding to/from PPP working for one netif, but not another

Andrew Pullin
Hi folks,

I am stuck on an issue here where I am trying to use lwip's
IP_FORWARDING feature.
I am trying to forward packets between a PPP server netif and a WAN
interface. It is working on case, where the WAN-connected device is
using WiFi, but then it fails to work in another case where the WAN
device is configured to use an Ethernet interface.

This is to ultimately bring Ethernet connectivity to a device without
Ethernet, but which can act as a PPP client (via UART).
Lwip is running on a device with Ethernet and using the lwip PPP server
implementation.

I have read the other threads involving PPP and forwarding, but I don't
believe I have seen quite this same problem arise.

Here is how the devices are arranged for each case:

Working:
    10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------> 
10.0.200.121 ----------> 10.0.1.1 ---> WAN
(PPP client)                    (PPP server, ESP32, lwip) (WiFi STA
netif)               ( router )

In this case, the WiFi netif reports the gateway IP correctly (10.0.1.1)
and the netmask as 255.255.0.0

Not working:
    10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------> 
10.0.200.113 ----------> 10.0.1.1 ---> WAN
(PPP client)                   (PPP server, ESP32, lwip)           
(Ethernet netif)               ( router )

In this case, the Ethernet netif reports the gateway IP correctly
(10.0.1.1) and the netmask as 255.255.0.0 , which appears to be the same
as for the working WiFi netif case.

The addresses 10.0.0.1 and 10.0.0.2 on each side of the PPP are manually
configured. They are hardcoded into the PPP server setup, and the client
appears to be able to take it's 10.0.0.2 address via IPCP over PPP.


To test that the WAN connection "works" for the PPP client, I am issuing
pings from PPP client device.
In the "working" case, I can ping 10.0.0.1, 10.0.1.1 (network gateway),
and 8.8.8.8. All work.
In the "not working" case on ethernet, I can ping 10.0.0.1 (just over
the PPP link), but 10.0.1.1 and 8.8.8.8 appear unreachable.

Of course, I am building with IP_FORWARD defined to 1. Debug logs do
reflect that forwarding is happening.
There are several shims in the Espressif SDK over the actual lwip netif
sets/calls.
As far as I can see, the same setup is done for the ethernet netif as is
for the wifi netif, but this is an obviously suspicious point.

In my case, I am using an ESP32 chip, and all of the lwip init and setup
is done for me in the SDK.
As far as I can see, the init for each adapter/netif type looks the same.

I have logs for each case, too:
Working (wifi netif) - https://pastebin.com/RgqHa5CR
Not working (ethernet netif) - https://pastebin.com/L80raiRF

One note here is: the PPP client is not running lwip. For hardware &
legacy reasons, it is running another network stack with a totally
different PPP client implementation.
But, given that the working/not working is a function of the PPP server
netif's, I did not suspect a mis-configuration issue in the PPP client.

Any insight here would be greatly appreciated. I am not too familiar
with the deep under-the-hood details of network stacks.

Thanks,
Andrew Pullin


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Sylvain Rochet
Hi Andrew,

On Tue, Jan 28, 2020 at 04:57:19PM -0800, Andrew Pullin wrote:

> Hi folks,
>
> I am stuck on an issue here where I am trying to use lwip's IP_FORWARDING
> feature.
> I am trying to forward packets between a PPP server netif and a WAN
> interface. It is working on case, where the WAN-connected device is using
> WiFi, but then it fails to work in another case where the WAN device is
> configured to use an Ethernet interface.
>
> This is to ultimately bring Ethernet connectivity to a device without
> Ethernet, but which can act as a PPP client (via UART).
> Lwip is running on a device with Ethernet and using the lwip PPP server
> implementation.
>
> I have read the other threads involving PPP and forwarding, but I don't
> believe I have seen quite this same problem arise.
>
> Here is how the devices are arranged for each case:
>
> Working:
>    10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------> 
> 10.0.200.121 ----------> 10.0.1.1 ---> WAN
> (PPP client)                    (PPP server, ESP32, lwip) (WiFi STA netif) 
>              ( router )
>
> In this case, the WiFi netif reports the gateway IP correctly (10.0.1.1) and
> the netmask as 255.255.0.0
>
> Not working:
>    10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------> 
> 10.0.200.113 ----------> 10.0.1.1 ---> WAN
> (PPP client)                   (PPP server, ESP32, lwip)           
> (Ethernet netif)               ( router )
>
> In this case, the Ethernet netif reports the gateway IP correctly (10.0.1.1)
> and the netmask as 255.255.0.0 , which appears to be the same as for the
> working WiFi netif case.
>
> The addresses 10.0.0.1 and 10.0.0.2 on each side of the PPP are manually
> configured. They are hardcoded into the PPP server setup, and the client
> appears to be able to take it's 10.0.0.2 address via IPCP over PPP.
>
>
> To test that the WAN connection "works" for the PPP client, I am issuing
> pings from PPP client device.
> In the "working" case, I can ping 10.0.0.1, 10.0.1.1 (network gateway), and
> 8.8.8.8. All work.
> In the "not working" case on ethernet, I can ping 10.0.0.1 (just over the
> PPP link), but 10.0.1.1 and 8.8.8.8 appear unreachable.
>
> Of course, I am building with IP_FORWARD defined to 1. Debug logs do reflect
> that forwarding is happening.
> There are several shims in the Espressif SDK over the actual lwip netif
> sets/calls.
> As far as I can see, the same setup is done for the ethernet netif as is for
> the wifi netif, but this is an obviously suspicious point.
>
> In my case, I am using an ESP32 chip, and all of the lwip init and setup is
> done for me in the SDK.
> As far as I can see, the init for each adapter/netif type looks the same.
>
> I have logs for each case, too:
> Working (wifi netif) - https://pastebin.com/RgqHa5CR
> Not working (ethernet netif) - https://pastebin.com/L80raiRF
>
> One note here is: the PPP client is not running lwip. For hardware & legacy
> reasons, it is running another network stack with a totally different PPP
> client implementation.
> But, given that the working/not working is a function of the PPP server
> netif's, I did not suspect a mis-configuration issue in the PPP client.
>
> Any insight here would be greatly appreciated. I am not too familiar with
> the deep under-the-hood details of network stacks.
I tried to understand but I failed. Before anything else, could you
share the real network configuration of all network interfaces, the
routing table on all hosts, and any speciafic features enabled on each
interface (NAT, ARP-Proxy, …), for all configurations you tried ?

Example:

[Host1]ppp0 --- ppp0[Host2]eth0 --- eth0[Host3]eth1 --- Internet

Host1:
  running a PPP client, that's all we know
  ppp0: 10.0.0.2/32
  default route: ppp0
  no other route

Host2:
  running lwIP
  ppp0: 10.0.0.1/32
  eth0: 10.0.200.121/16
  default route: 10.0.1.1
  no other route

Host3:
  gateway ? to what ?
  eth0: 10.0.1.1/16
  eth1: public IPv4, facing Internet, set by DHCP, SNAT
  default route: set by DHCP, toward eth1
  no other route

Hint: the configuration above can't work, but I'm not sure this is what
you currently have...

Sylvain

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Andrew Pullin
On 1/29/20 5:42 AM, Sylvain Rochet wrote:

> Hi Andrew,
>
> On Tue, Jan 28, 2020 at 04:57:19PM -0800, Andrew Pullin wrote:
>> Hi folks,
>>
>> I am stuck on an issue here where I am trying to use lwip's IP_FORWARDING
>> feature.
>> I am trying to forward packets between a PPP server netif and a WAN
>> interface. It is working on case, where the WAN-connected device is using
>> WiFi, but then it fails to work in another case where the WAN device is
>> configured to use an Ethernet interface.
>>
>> This is to ultimately bring Ethernet connectivity to a device without
>> Ethernet, but which can act as a PPP client (via UART).
>> Lwip is running on a device with Ethernet and using the lwip PPP server
>> implementation.
>>
>> I have read the other threads involving PPP and forwarding, but I don't
>> believe I have seen quite this same problem arise.
>>
>> Here is how the devices are arranged for each case:
>>
>> Working:
>>     10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------>
>> 10.0.200.121 ----------> 10.0.1.1 ---> WAN
>> (PPP client)                    (PPP server, ESP32, lwip) (WiFi STA netif)
>>               ( router )
>>
>> In this case, the WiFi netif reports the gateway IP correctly (10.0.1.1) and
>> the netmask as 255.255.0.0
>>
>> Not working:
>>     10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------>
>> 10.0.200.113 ----------> 10.0.1.1 ---> WAN
>> (PPP client)                   (PPP server, ESP32, lwip)
>> (Ethernet netif)               ( router )
>>
>> In this case, the Ethernet netif reports the gateway IP correctly (10.0.1.1)
>> and the netmask as 255.255.0.0 , which appears to be the same as for the
>> working WiFi netif case.
>>
>> The addresses 10.0.0.1 and 10.0.0.2 on each side of the PPP are manually
>> configured. They are hardcoded into the PPP server setup, and the client
>> appears to be able to take it's 10.0.0.2 address via IPCP over PPP.
>>
>>
>> To test that the WAN connection "works" for the PPP client, I am issuing
>> pings from PPP client device.
>> In the "working" case, I can ping 10.0.0.1, 10.0.1.1 (network gateway), and
>> 8.8.8.8. All work.
>> In the "not working" case on ethernet, I can ping 10.0.0.1 (just over the
>> PPP link), but 10.0.1.1 and 8.8.8.8 appear unreachable.
>>
>> Of course, I am building with IP_FORWARD defined to 1. Debug logs do reflect
>> that forwarding is happening.
>> There are several shims in the Espressif SDK over the actual lwip netif
>> sets/calls.
>> As far as I can see, the same setup is done for the ethernet netif as is for
>> the wifi netif, but this is an obviously suspicious point.
>>
>> In my case, I am using an ESP32 chip, and all of the lwip init and setup is
>> done for me in the SDK.
>> As far as I can see, the init for each adapter/netif type looks the same.
>>
>> I have logs for each case, too:
>> Working (wifi netif) - https://pastebin.com/RgqHa5CR
>> Not working (ethernet netif) - https://pastebin.com/L80raiRF
>>
>> One note here is: the PPP client is not running lwip. For hardware & legacy
>> reasons, it is running another network stack with a totally different PPP
>> client implementation.
>> But, given that the working/not working is a function of the PPP server
>> netif's, I did not suspect a mis-configuration issue in the PPP client.
>>
>> Any insight here would be greatly appreciated. I am not too familiar with
>> the deep under-the-hood details of network stacks.
> I tried to understand but I failed. Before anything else, could you
> share the real network configuration of all network interfaces, the
> routing table on all hosts, and any speciafic features enabled on each
> interface (NAT, ARP-Proxy, …), for all configurations you tried ?
>
> Example:
>
> [Host1]ppp0 --- ppp0[Host2]eth0 --- eth0[Host3]eth1 --- Internet
>
> Host1:
>    running a PPP client, that's all we know
>    ppp0: 10.0.0.2/32
>    default route: ppp0
>    no other route
>
> Host2:
>    running lwIP
>    ppp0: 10.0.0.1/32
>    eth0: 10.0.200.121/16
>    default route: 10.0.1.1
>    no other route
>
> Host3:
>    gateway ? to what ?
>    eth0: 10.0.1.1/16
>    eth1: public IPv4, facing Internet, set by DHCP, SNAT
>    default route: set by DHCP, toward eth1
>    no other route
>
> Hint: the configuration above can't work, but I'm not sure this is what
> you currently have...
>
> Sylvain
Uh-oh, I think I did a poor job of explaining it. Although you seem to
have surmised most of it.

The full rundown:

Host1:
     - Micontroller device with no MAC or PHY
     - running NetX IP stack
     - Runs a PPP client, specifically PPPoS using TTL UART
     - default route: ppp0
     - ppp0 is the only network interface available
     - ARP is supported and enabled
     - IP 10.0.0.2 is assigned during PPP negotiation

Host2:
     - Micontroller device, ESP32
     - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY),
used mutually exclusively
     - Running lwIP
     - WiFi and Ethernet are independently verified to work to reach WAN
from this host
     - PPPoS server
     - ppp0 : 10.0.0.1/32
         - IP for server and client are hard-coded in
         - I believe it is /32, as the netmask is reported as
255.255.255.255
     - When running w/ WiFi:
         - wlan0 : ip 10.0.200.136/16  (IP and /16 from DHCP server)
     - When running w/ Ethernet:
         - eth0 : ip 10.0.200.113/16  (IP and /16 from DHCP server)
     - Routing is unknown. lwIP built with IP_FORWARD enabled.
     - default route: wlan0 or eth0 (assumed)
     - ARP is enabled.

WiFi AP's:
     - AP only (no routing)
     - Connected to Host3 by Ethernet
     - MACs 0c:8d:db:6e:f0:03 or 0c:8d:db:6e:f0:88 (can't control
association)

Host3:
     - Router (pfSense)
     - DHCP server runs here
     - Routes to WAN, does NAT, eth0 <-> eth1
     - default route: set by DHCP, toward eth1
     - no other route (as far as I know)
     - Intranet WiFi AP's are connected via switch to eth0
     - Intranet Ethernet devices are connected via switch to eth0
     - eth0 : 10.0.1.1/16 , facing intranet, MAC 00:e0:67:18:54:95
     - eth1 : IP from ISP, facing public IPv4 internet, MAC
00:e0:67:18:54:94

Layout:

Host1 <-----------------> Host2 <-------------------------> Host3
<---------> public internet
                 PPPoS                             Ethernet

or

Host1 <-----------------> Host2 <---------------->  WiFi AP
<------------------> Host3 <-------> public internet
                 PPPoS WiFi                              Ethernet


For the case of using the WiFi connection, there are separate WiFi AP's
in the way.
Other than that, the two cases are host2 connecting to the local subnet
via wlan0 or connecting via eth0.

I am not manually adding any static routes in either Host1 or Host2.
I was worried that Host2 would not be able to accomplish the "connection
sharing" until I found the IP_FORWARDING option. That initially worked
with Host2 using WiFi without any manually added static routes.

Unless I am missing something being changed in the config, that is the
config both both the case of Host2 using wlan0 and for using eth0. The
only change I can observe is that host2 IP is given a different IP by
host3 DHCP.

Very interesting that you already see something wrong ... that means I
am likely missing something obvious. I certainly hope it is something
simple to tweak to make this work over host2's eth0!

Unfortunately, I have no ability to do debug on the ESP32, so I cannot
check how the stack is really behaving when checking if anything needs
to be forward.
Using the ETHARP_TRUST option does not appear to have an effect, either.
I did not enable Proxy ARP in either case, and it doesn't look like the
drivers around netif enable it, either.
As far as I know, NAT is only running in Host3, which is the router
between the whole subnet and the public internet.

I regenerated the logs with ARP logging and with relative timestamps, in
case that helps:

Host2 using wlan0, host1 is able to ping 8.8.8.8 and 10.0.1.1 :
https://pastebin.com/GJkSsPbb
Host2 using eth0, host1 is not able to ping 8.8.8.8 and 10.0.1.1 :
https://pastebin.com/riE345HN

The help is really appreciated.

Thanks,
Andrew Pullin


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Jens Nielsen-2
Hi,

On 2020-01-30 05:15, Andrew Pullin wrote:

> On 1/29/20 5:42 AM, Sylvain Rochet wrote:
>> Hi Andrew,
>>
>> On Tue, Jan 28, 2020 at 04:57:19PM -0800, Andrew Pullin wrote:
>>> Hi folks,
>>>
>>> I am stuck on an issue here where I am trying to use lwip's
>>> IP_FORWARDING
>>> feature.
>>> I am trying to forward packets between a PPP server netif and a WAN
>>> interface. It is working on case, where the WAN-connected device is
>>> using
>>> WiFi, but then it fails to work in another case where the WAN device is
>>> configured to use an Ethernet interface.
>>>
>>> This is to ultimately bring Ethernet connectivity to a device without
>>> Ethernet, but which can act as a PPP client (via UART).
>>> Lwip is running on a device with Ethernet and using the lwip PPP server
>>> implementation.
>>>
>>> I have read the other threads involving PPP and forwarding, but I don't
>>> believe I have seen quite this same problem arise.
>>>
>>> Here is how the devices are arranged for each case:
>>>
>>> Working:
>>>     10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------>
>>> 10.0.200.121 ----------> 10.0.1.1 ---> WAN
>>> (PPP client)                    (PPP server, ESP32, lwip) (WiFi STA
>>> netif)
>>>               ( router )
>>>
>>> In this case, the WiFi netif reports the gateway IP correctly
>>> (10.0.1.1) and
>>> the netmask as 255.255.0.0
>>>
>>> Not working:
>>>     10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------>
>>> 10.0.200.113 ----------> 10.0.1.1 ---> WAN
>>> (PPP client)                   (PPP server, ESP32, lwip)
>>> (Ethernet netif)               ( router )
>>>
>>> In this case, the Ethernet netif reports the gateway IP correctly
>>> (10.0.1.1)
>>> and the netmask as 255.255.0.0 , which appears to be the same as for
>>> the
>>> working WiFi netif case.
>>>
>>> The addresses 10.0.0.1 and 10.0.0.2 on each side of the PPP are
>>> manually
>>> configured. They are hardcoded into the PPP server setup, and the
>>> client
>>> appears to be able to take it's 10.0.0.2 address via IPCP over PPP.
>>>
>>>
>>> To test that the WAN connection "works" for the PPP client, I am
>>> issuing
>>> pings from PPP client device.
>>> In the "working" case, I can ping 10.0.0.1, 10.0.1.1 (network
>>> gateway), and
>>> 8.8.8.8. All work.
>>> In the "not working" case on ethernet, I can ping 10.0.0.1 (just
>>> over the
>>> PPP link), but 10.0.1.1 and 8.8.8.8 appear unreachable.
>>>
>>> Of course, I am building with IP_FORWARD defined to 1. Debug logs do
>>> reflect
>>> that forwarding is happening.
>>> There are several shims in the Espressif SDK over the actual lwip netif
>>> sets/calls.
>>> As far as I can see, the same setup is done for the ethernet netif
>>> as is for
>>> the wifi netif, but this is an obviously suspicious point.
>>>
>>> In my case, I am using an ESP32 chip, and all of the lwip init and
>>> setup is
>>> done for me in the SDK.
>>> As far as I can see, the init for each adapter/netif type looks the
>>> same.
>>>
>>> I have logs for each case, too:
>>> Working (wifi netif) - https://pastebin.com/RgqHa5CR
>>> Not working (ethernet netif) - https://pastebin.com/L80raiRF
>>>
>>> One note here is: the PPP client is not running lwip. For hardware &
>>> legacy
>>> reasons, it is running another network stack with a totally
>>> different PPP
>>> client implementation.
>>> But, given that the working/not working is a function of the PPP server
>>> netif's, I did not suspect a mis-configuration issue in the PPP client.
>>>
>>> Any insight here would be greatly appreciated. I am not too familiar
>>> with
>>> the deep under-the-hood details of network stacks.
>> I tried to understand but I failed. Before anything else, could you
>> share the real network configuration of all network interfaces, the
>> routing table on all hosts, and any speciafic features enabled on each
>> interface (NAT, ARP-Proxy, …), for all configurations you tried ?
>>
>> Example:
>>
>> [Host1]ppp0 --- ppp0[Host2]eth0 --- eth0[Host3]eth1 --- Internet
>>
>> Host1:
>>    running a PPP client, that's all we know
>>    ppp0: 10.0.0.2/32
>>    default route: ppp0
>>    no other route
>>
>> Host2:
>>    running lwIP
>>    ppp0: 10.0.0.1/32
>>    eth0: 10.0.200.121/16
>>    default route: 10.0.1.1
>>    no other route
>>
>> Host3:
>>    gateway ? to what ?
>>    eth0: 10.0.1.1/16
>>    eth1: public IPv4, facing Internet, set by DHCP, SNAT
>>    default route: set by DHCP, toward eth1
>>    no other route
>>
>> Hint: the configuration above can't work, but I'm not sure this is what
>> you currently have...
>>
>> Sylvain
> Uh-oh, I think I did a poor job of explaining it. Although you seem to
> have surmised most of it.
>
> The full rundown:
>
> Host1:
>     - Micontroller device with no MAC or PHY
>     - running NetX IP stack
>     - Runs a PPP client, specifically PPPoS using TTL UART
>     - default route: ppp0
>     - ppp0 is the only network interface available
>     - ARP is supported and enabled
>     - IP 10.0.0.2 is assigned during PPP negotiation
>
> Host2:
>     - Micontroller device, ESP32
>     - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY),
> used mutually exclusively
>     - Running lwIP
>     - WiFi and Ethernet are independently verified to work to reach
> WAN from this host
>     - PPPoS server
>     - ppp0 : 10.0.0.1/32
>         - IP for server and client are hard-coded in
>         - I believe it is /32, as the netmask is reported as
> 255.255.255.255
>     - When running w/ WiFi:
>         - wlan0 : ip 10.0.200.136/16  (IP and /16 from DHCP server)
>     - When running w/ Ethernet:
>         - eth0 : ip 10.0.200.113/16  (IP and /16 from DHCP server)
>     - Routing is unknown. lwIP built with IP_FORWARD enabled.
>     - default route: wlan0 or eth0 (assumed)
>     - ARP is enabled.
>
> WiFi AP's:
>     - AP only (no routing)
>     - Connected to Host3 by Ethernet
>     - MACs 0c:8d:db:6e:f0:03 or 0c:8d:db:6e:f0:88 (can't control
> association)
>
> Host3:
>     - Router (pfSense)
>     - DHCP server runs here
>     - Routes to WAN, does NAT, eth0 <-> eth1
>     - default route: set by DHCP, toward eth1
>     - no other route (as far as I know)
>     - Intranet WiFi AP's are connected via switch to eth0
>     - Intranet Ethernet devices are connected via switch to eth0
>     - eth0 : 10.0.1.1/16 , facing intranet, MAC 00:e0:67:18:54:95
>     - eth1 : IP from ISP, facing public IPv4 internet, MAC
> 00:e0:67:18:54:94
>
> Layout:
>
> Host1 <-----------------> Host2 <-------------------------> Host3
> <---------> public internet
>                 PPPoS                             Ethernet
>
> or
>
> Host1 <-----------------> Host2 <----------------> WiFi AP
> <------------------> Host3 <-------> public internet
>                 PPPoS WiFi                              Ethernet
>
>
> For the case of using the WiFi connection, there are separate WiFi
> AP's in the way.
> Other than that, the two cases are host2 connecting to the local
> subnet via wlan0 or connecting via eth0.
>
> I am not manually adding any static routes in either Host1 or Host2.
> I was worried that Host2 would not be able to accomplish the
> "connection sharing" until I found the IP_FORWARDING option. That
> initially worked with Host2 using WiFi without any manually added
> static routes.
>
> Unless I am missing something being changed in the config, that is the
> config both both the case of Host2 using wlan0 and for using eth0. The
> only change I can observe is that host2 IP is given a different IP by
> host3 DHCP.
>
> Very interesting that you already see something wrong ... that means I
> am likely missing something obvious. I certainly hope it is something
> simple to tweak to make this work over host2's eth0!
>
> Unfortunately, I have no ability to do debug on the ESP32, so I cannot
> check how the stack is really behaving when checking if anything needs
> to be forward.
> Using the ETHARP_TRUST option does not appear to have an effect, either.
> I did not enable Proxy ARP in either case, and it doesn't look like
> the drivers around netif enable it, either.
> As far as I know, NAT is only running in Host3, which is the router
> between the whole subnet and the public internet.
>
> I regenerated the logs with ARP logging and with relative timestamps,
> in case that helps:
>
> Host2 using wlan0, host1 is able to ping 8.8.8.8 and 10.0.1.1 :
> https://pastebin.com/GJkSsPbb
> Host2 using eth0, host1 is not able to ping 8.8.8.8 and 10.0.1.1 :
> https://pastebin.com/riE345HN
>
> The help is really appreciated.
>
> Thanks,
> Andrew Pullin
>
>
> _______________________________________________
> lwip-users mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/lwip-users


Could you try changing the ip addresses of your ppp network so they're
not on the 10.0.0.0/16 subnet? Maybe use 10.1.0.1 and 10.1.0.2?

I don't know why it would work in one case and not the other but afaik
overlapping subnets isn't supported

You say "default route: wlan0 or eth0 (assumed)" but are you also
setting the corresponding netif as default?
(netifapi_netif_set_default/netif_set_default)

Best regards
Jens


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Sylvain Rochet
In reply to this post by Andrew Pullin
Hi Andrew,

On Wed, Jan 29, 2020 at 08:15:11PM -0800, Andrew Pullin wrote:

>
> Uh-oh, I think I did a poor job of explaining it. Although you seem to have
> surmised most of it.
>
> The full rundown:
>
> Host1:
>     - Micontroller device with no MAC or PHY
>     - running NetX IP stack
>     - Runs a PPP client, specifically PPPoS using TTL UART
>     - default route: ppp0
>     - ppp0 is the only network interface available
>     - ARP is supported and enabled
>     - IP 10.0.0.2 is assigned during PPP negotiation
There are no ARP in PPP, but whatever, this configuration is correct.


> Host2:
>     - Micontroller device, ESP32
>     - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used
> mutually exclusively
>     - Running lwIP
>     - WiFi and Ethernet are independently verified to work to reach WAN from
> this host
>     - PPPoS server
>     - ppp0 : 10.0.0.1/32
>         - IP for server and client are hard-coded in
>         - I believe it is /32, as the netmask is reported as 255.255.255.255
>     - When running w/ WiFi:
>         - wlan0 : ip 10.0.200.136/16  (IP and /16 from DHCP server)
>     - When running w/ Ethernet:
>         - eth0 : ip 10.0.200.113/16  (IP and /16 from DHCP server)
>     - Routing is unknown. lwIP built with IP_FORWARD enabled.
>     - default route: wlan0 or eth0 (assumed)
>     - ARP is enabled.
10.0.0.1/32 and 10.0.0.2/32 are within 10.0.0.0/16 subnet, that
shouldn't be a problem here, this is usually a bad practice and it
should be avoided. This is a problem for hosts on 10.0.0.0/16 that don't
know there is a hole in the subnet, since there are only 3 hosts here,
which is an expection rather than the rule, it is safe by luck.


> WiFi AP's:
>     - AP only (no routing)
>     - Connected to Host3 by Ethernet
>     - MACs 0c:8d:db:6e:f0:03 or 0c:8d:db:6e:f0:88 (can't control
> association)
>
> Host3:
>     - Router (pfSense)
>     - DHCP server runs here
>     - Routes to WAN, does NAT, eth0 <-> eth1
>     - default route: set by DHCP, toward eth1
>     - no other route (as far as I know)
>     - Intranet WiFi AP's are connected via switch to eth0
>     - Intranet Ethernet devices are connected via switch to eth0
>     - eth0 : 10.0.1.1/16 , facing intranet, MAC 00:e0:67:18:54:95
>     - eth1 : IP from ISP, facing public IPv4 internet, MAC 00:e0:67:18:54:94
Here is the real problem, you need a route on Host3 telling that
10.0.0.2/32 is behind 10.0.200.136 (or 10.0.200.113). You can also add a
route for 10.0.0.1/32 but it's just a bonus.

Without this route, Host3 don't know how to route the packet and will
try to find an host on the 10.0.0.0/16 subnet using ARP. Then it's just
luck, and depends on what Host3 and Host2 are doing:

- If the packet is sent in a broadcast Ethernet frame by Host3 it might
works if Host2 catches the packet and forwards it to Host1.

- If the ARP request gets answered by Host2 when Host3 send their ARP
request to find who is 10.0.0.2 on the 10.0.0.0/16 subnet, then it also
works. This is known as ARP Proxy, lwIP does not support that.

It's just luck, using lazy features of IP stack to make it works even if
the configuration is broken should be avoided :-)


> Layout:
>
> Host1 <-----------------> Host2 <-------------------------> Host3
> <---------> public internet
>                 PPPoS                             Ethernet
>
> or
>
> Host1 <-----------------> Host2 <---------------->  WiFi AP
> <------------------> Host3 <-------> public internet
>                 PPPoS WiFi                              Ethernet
>
>
> For the case of using the WiFi connection, there are separate WiFi AP's in
> the way.
> Other than that, the two cases are host2 connecting to the local subnet via
> wlan0 or connecting via eth0.
I guess the only reason it works with WiFi and not with Ethernet is
because WiFi is intrinsically broadcast-based while Ethernet is switched
(well, I hope so, unless you live in '90s :p). This is the "works
because Host2 catches the broadcast packet" case I've said before.


> I am not manually adding any static routes in either Host1 or Host2.
> I was worried that Host2 would not be able to accomplish the "connection
> sharing" until I found the IP_FORWARDING option. That initially worked with
> Host2 using WiFi without any manually added static routes.

This is not "connection sharing", this is IP routing, not NAT !,
"connection sharing" is usually used for hosts that do SNAT on their
Interface facing Internet.

IP routing needs a complete routing table on all hosts telling how to
reach (i.e. which next-hop to use) every host of the whole network.

Example with 2 subnets and 3 Hosts:

A-eth0  --------------------- eth0-B-eth1 --------------------- eth0-C
  192.168.0.1/24    192.168.0.2/24   172.16.1.1/24     172.16.1.2/24

Host A routing table:
192.168.0.0/24 dev eth0
172.16.1.0/24 via 192.168.0.2

Host B routing table:
192.168.0.0/24 dev eth0
172.16.1.0/24 dev eth1

Host C routing table:
192.168.0.0/24 via 172.16.1.1
172.16.1.0/24 dev eth0

dev = directly connected
via = next-hop, gateway if you prefer

What you could see is that each host have a similar routing table, just
the next-hop changes. Default route is just a way to reduce the routing
table size by removing all entries that have an identical next-hop.


> Unless I am missing something being changed in the config, that is the
> config both both the case of Host2 using wlan0 and for using eth0. The only
> change I can observe is that host2 IP is given a different IP by host3 DHCP.
>
> Very interesting that you already see something wrong ... that means I am
> likely missing something obvious. I certainly hope it is something simple to
> tweak to make this work over host2's eth0!
>
> Unfortunately, I have no ability to do debug on the ESP32, so I cannot check
> how the stack is really behaving when checking if anything needs to be
> forward.
> Using the ETHARP_TRUST option does not appear to have an effect, either.
> I did not enable Proxy ARP in either case, and it doesn't look like the
> drivers around netif enable it, either.
> As far as I know, NAT is only running in Host3, which is the router between
> the whole subnet and the public internet.
>
> I regenerated the logs with ARP logging and with relative timestamps, in
> case that helps:
>
> Host2 using wlan0, host1 is able to ping 8.8.8.8 and 10.0.1.1 :
> https://pastebin.com/GJkSsPbb
> Host2 using eth0, host1 is not able to ping 8.8.8.8 and 10.0.1.1 :
> https://pastebin.com/riE345HN
>
> The help is really appreciated.

Sylvain

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

goldsimon@gmx.de
Am 30.01.2020 um 20:42 schrieb Sylvain Rochet:

> Hi Andrew,
>
> On Wed, Jan 29, 2020 at 08:15:11PM -0800, Andrew Pullin wrote:
>>
>> Uh-oh, I think I did a poor job of explaining it. Although you seem to have
>> surmised most of it.
>>
>> The full rundown:
>>
>> Host1:
>>      - Micontroller device with no MAC or PHY
>>      - running NetX IP stack
>>      - Runs a PPP client, specifically PPPoS using TTL UART
>>      - default route: ppp0
>>      - ppp0 is the only network interface available
>>      - ARP is supported and enabled
>>      - IP 10.0.0.2 is assigned during PPP negotiation
>
> There are no ARP in PPP, but whatever, this configuration is correct.
>
>
>> Host2:
>>      - Micontroller device, ESP32
>>      - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used
>> mutually exclusively
>>      - Running lwIP
>>      - WiFi and Ethernet are independently verified to work to reach WAN from
>> this host
>>      - PPPoS server
>>      - ppp0 : 10.0.0.1/32
>>          - IP for server and client are hard-coded in
>>          - I believe it is /32, as the netmask is reported as 255.255.255.255
>>      - When running w/ WiFi:
>>          - wlan0 : ip 10.0.200.136/16  (IP and /16 from DHCP server)
>>      - When running w/ Ethernet:
>>          - eth0 : ip 10.0.200.113/16  (IP and /16 from DHCP server)

One odd thing I find here is that lwIP does not support 2 interfaces in
the same subnet. So you would have to take care that only one netif at a
time has its link set up (or its address set or its administrative
status up, or whatever). In any case, having both wlan0 and eth0 up and
running with an address in the same subnet will result in packets being
accepted on both netifs but transmitted on whichever was registered last
(or was it first?).

>>      - Routing is unknown. lwIP built with IP_FORWARD enabled.
>>      - default route: wlan0 or eth0 (assumed)
>>      - ARP is enabled.
>
> 10.0.0.1/32 and 10.0.0.2/32 are within 10.0.0.0/16 subnet, that
> shouldn't be a problem here, this is usually a bad practice and it
> should be avoided. This is a problem for hosts on 10.0.0.0/16 that don't
> know there is a hole in the subnet, since there are only 3 hosts here,
> which is an expection rather than the rule, it is safe by luck.
>
>
>> WiFi AP's:
>>      - AP only (no routing)
>>      - Connected to Host3 by Ethernet
>>      - MACs 0c:8d:db:6e:f0:03 or 0c:8d:db:6e:f0:88 (can't control
>> association)
>>
>> Host3:
>>      - Router (pfSense)
>>      - DHCP server runs here
>>      - Routes to WAN, does NAT, eth0 <-> eth1
>>      - default route: set by DHCP, toward eth1
>>      - no other route (as far as I know)
>>      - Intranet WiFi AP's are connected via switch to eth0
>>      - Intranet Ethernet devices are connected via switch to eth0
>>      - eth0 : 10.0.1.1/16 , facing intranet, MAC 00:e0:67:18:54:95
>>      - eth1 : IP from ISP, facing public IPv4 internet, MAC 00:e0:67:18:54:94
>
> Here is the real problem, you need a route on Host3 telling that
> 10.0.0.2/32 is behind 10.0.200.136 (or 10.0.200.113). You can also add a
> route for 10.0.0.1/32 but it's just a bonus.
>
> Without this route, Host3 don't know how to route the packet and will
> try to find an host on the 10.0.0.0/16 subnet using ARP. Then it's just
> luck, and depends on what Host3 and Host2 are doing:
>
> - If the packet is sent in a broadcast Ethernet frame by Host3 it might
> works if Host2 catches the packet and forwards it to Host1.
>
> - If the ARP request gets answered by Host2 when Host3 send their ARP
> request to find who is 10.0.0.2 on the 10.0.0.0/16 subnet, then it also
> works. This is known as ARP Proxy, lwIP does not support that.
>
> It's just luck, using lazy features of IP stack to make it works even if
> the configuration is broken should be avoided :-)
>
>
>> Layout:
>>
>> Host1 <-----------------> Host2 <-------------------------> Host3
>> <---------> public internet
>>                  PPPoS                             Ethernet
>>
>> or
>>
>> Host1 <-----------------> Host2 <---------------->  WiFi AP
>> <------------------> Host3 <-------> public internet
>>                  PPPoS WiFi                              Ethernet
>>
>>
>> For the case of using the WiFi connection, there are separate WiFi AP's in
>> the way.
>> Other than that, the two cases are host2 connecting to the local subnet via
>> wlan0 or connecting via eth0.
>
> I guess the only reason it works with WiFi and not with Ethernet is
> because WiFi is intrinsically broadcast-based while Ethernet is switched
> (well, I hope so, unless you live in '90s :p). This is the "works
> because Host2 catches the broadcast packet" case I've said before.
>
>
>> I am not manually adding any static routes in either Host1 or Host2.
>> I was worried that Host2 would not be able to accomplish the "connection
>> sharing" until I found the IP_FORWARDING option. That initially worked with
>> Host2 using WiFi without any manually added static routes.
>
> This is not "connection sharing", this is IP routing, not NAT !,
> "connection sharing" is usually used for hosts that do SNAT on their
> Interface facing Internet.
>
> IP routing needs a complete routing table on all hosts telling how to
> reach (i.e. which next-hop to use) every host of the whole network.
>
> Example with 2 subnets and 3 Hosts:
>
> A-eth0  --------------------- eth0-B-eth1 --------------------- eth0-C
>    192.168.0.1/24    192.168.0.2/24   172.16.1.1/24     172.16.1.2/24
>
> Host A routing table:
> 192.168.0.0/24 dev eth0
> 172.16.1.0/24 via 192.168.0.2
>
> Host B routing table:
> 192.168.0.0/24 dev eth0
> 172.16.1.0/24 dev eth1
>
> Host C routing table:
> 192.168.0.0/24 via 172.16.1.1
> 172.16.1.0/24 dev eth0
>
> dev = directly connected
> via = next-hop, gateway if you prefer
>
> What you could see is that each host have a similar routing table, just
> the next-hop changes. Default route is just a way to reduce the routing
> table size by removing all entries that have an identical next-hop.

Which brings me to a question: how do we define such routing rules in
lwIP? Hmm...

Regards,
Simon

>
>
>> Unless I am missing something being changed in the config, that is the
>> config both both the case of Host2 using wlan0 and for using eth0. The only
>> change I can observe is that host2 IP is given a different IP by host3 DHCP.
>>
>> Very interesting that you already see something wrong ... that means I am
>> likely missing something obvious. I certainly hope it is something simple to
>> tweak to make this work over host2's eth0!
>>
>> Unfortunately, I have no ability to do debug on the ESP32, so I cannot check
>> how the stack is really behaving when checking if anything needs to be
>> forward.
>> Using the ETHARP_TRUST option does not appear to have an effect, either.
>> I did not enable Proxy ARP in either case, and it doesn't look like the
>> drivers around netif enable it, either.
>> As far as I know, NAT is only running in Host3, which is the router between
>> the whole subnet and the public internet.
>>
>> I regenerated the logs with ARP logging and with relative timestamps, in
>> case that helps:
>>
>> Host2 using wlan0, host1 is able to ping 8.8.8.8 and 10.0.1.1 :
>> https://pastebin.com/GJkSsPbb
>> Host2 using eth0, host1 is not able to ping 8.8.8.8 and 10.0.1.1 :
>> https://pastebin.com/riE345HN
>>
>> The help is really appreciated.
>
>
> Sylvain
>
>
> _______________________________________________
> lwip-users mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/lwip-users
>


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Sylvain Rochet
Hi Simon,

On Thu, Jan 30, 2020 at 09:31:11PM +0100, [hidden email] wrote:

> Am 30.01.2020 um 20:42 schrieb Sylvain Rochet:
> > On Wed, Jan 29, 2020 at 08:15:11PM -0800, Andrew Pullin wrote:
> > >
> > > Host2:
> > >     - Micontroller device, ESP32
> > >     - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used
> > >mutually exclusively
> > >     - Running lwIP
> > >     - WiFi and Ethernet are independently verified to work to reach WAN from
> > >this host
> > >     - PPPoS server
> > >     - ppp0 : 10.0.0.1/32
> > >         - IP for server and client are hard-coded in
> > >         - I believe it is /32, as the netmask is reported as 255.255.255.255
> > >     - When running w/ WiFi:
> > >         - wlan0 : ip 10.0.200.136/16  (IP and /16 from DHCP server)
> > >     - When running w/ Ethernet:
> > >         - eth0 : ip 10.0.200.113/16  (IP and /16 from DHCP server)
>
> One odd thing I find here is that lwIP does not support 2 interfaces in
> the same subnet. So you would have to take care that only one netif at a
> time has its link set up (or its address set or its administrative
> status up, or whatever). In any case, having both wlan0 and eth0 up and
> running with an address in the same subnet will result in packets being
> accepted on both netifs but transmitted on whichever was registered last
> (or was it first?).
I assumed that both were not active at the same time, good catch, this
might be an issue as well.


> > This is not "connection sharing", this is IP routing, not NAT !,
> > "connection sharing" is usually used for hosts that do SNAT on their
> > Interface facing Internet.
> >
> > IP routing needs a complete routing table on all hosts telling how to
> > reach (i.e. which next-hop to use) every host of the whole network.
> >
> > Example with 2 subnets and 3 Hosts:
> >
> > A-eth0  --------------------- eth0-B-eth1 --------------------- eth0-C
> >    192.168.0.1/24    192.168.0.2/24   172.16.1.1/24     172.16.1.2/24
> >
> > Host A routing table:
> > 192.168.0.0/24 dev eth0
> > 172.16.1.0/24 via 192.168.0.2
> >
> > Host B routing table:
> > 192.168.0.0/24 dev eth0
> > 172.16.1.0/24 dev eth1
> >
> > Host C routing table:
> > 192.168.0.0/24 via 172.16.1.1
> > 172.16.1.0/24 dev eth0
> >
> > dev = directly connected
> > via = next-hop, gateway if you prefer
> >
> > What you could see is that each host have a similar routing table, just
> > the next-hop changes. Default route is just a way to reduce the routing
> > table size by removing all entries that have an identical next-hop.
>
> Which brings me to a question: how do we define such routing rules in
> lwIP? Hmm...
I'm surprised by the question :-) In lwIP, "dev" rules don't need to be
added, the ip4_route function "create" them dynamically using the
interface address/netmask.

But thanks you for asking that because there is a huuuuuge issue about
that with the aforementioned case. Since "dev" rules are not ordered
because we just foreach on interfaces list until a match, we don't
respect that smaller subnets should be checked and matched first. So if
an IP subnet is within the IP subnet of another interface, although
perfectly legal configuration but *bad* design, it does not work in
lwIP. In lwIP it actually depends on the order interfaces were added :-)

For "via" routes, we have LWIP_HOOK_IP4_ROUTE.

Sylvain

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

goldsimon@gmx.de
Am 30.01.2020 um 22:02 schrieb Sylvain Rochet:

> Hi Simon,
>
> On Thu, Jan 30, 2020 at 09:31:11PM +0100, [hidden email] wrote:
>> Am 30.01.2020 um 20:42 schrieb Sylvain Rochet:
>>> On Wed, Jan 29, 2020 at 08:15:11PM -0800, Andrew Pullin wrote:
>>>>
>>>> Host2:
>>>>      - Micontroller device, ESP32
>>>>      - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used
>>>> mutually exclusively
>>>>      - Running lwIP
>>>>      - WiFi and Ethernet are independently verified to work to reach WAN from
>>>> this host
>>>>      - PPPoS server
>>>>      - ppp0 : 10.0.0.1/32
>>>>          - IP for server and client are hard-coded in
>>>>          - I believe it is /32, as the netmask is reported as 255.255.255.255
>>>>      - When running w/ WiFi:
>>>>          - wlan0 : ip 10.0.200.136/16  (IP and /16 from DHCP server)
>>>>      - When running w/ Ethernet:
>>>>          - eth0 : ip 10.0.200.113/16  (IP and /16 from DHCP server)
>>
>> One odd thing I find here is that lwIP does not support 2 interfaces in
>> the same subnet. So you would have to take care that only one netif at a
>> time has its link set up (or its address set or its administrative
>> status up, or whatever). In any case, having both wlan0 and eth0 up and
>> running with an address in the same subnet will result in packets being
>> accepted on both netifs but transmitted on whichever was registered last
>> (or was it first?).
>
> I assumed that both were not active at the same time, good catch, this
> might be an issue as well.
>
>
>>> This is not "connection sharing", this is IP routing, not NAT !,
>>> "connection sharing" is usually used for hosts that do SNAT on their
>>> Interface facing Internet.
>>>
>>> IP routing needs a complete routing table on all hosts telling how to
>>> reach (i.e. which next-hop to use) every host of the whole network.
>>>
>>> Example with 2 subnets and 3 Hosts:
>>>
>>> A-eth0  --------------------- eth0-B-eth1 --------------------- eth0-C
>>>     192.168.0.1/24    192.168.0.2/24   172.16.1.1/24     172.16.1.2/24
>>>
>>> Host A routing table:
>>> 192.168.0.0/24 dev eth0
>>> 172.16.1.0/24 via 192.168.0.2
>>>
>>> Host B routing table:
>>> 192.168.0.0/24 dev eth0
>>> 172.16.1.0/24 dev eth1
>>>
>>> Host C routing table:
>>> 192.168.0.0/24 via 172.16.1.1
>>> 172.16.1.0/24 dev eth0
>>>
>>> dev = directly connected
>>> via = next-hop, gateway if you prefer
>>>
>>> What you could see is that each host have a similar routing table, just
>>> the next-hop changes. Default route is just a way to reduce the routing
>>> table size by removing all entries that have an identical next-hop.
>>
>> Which brings me to a question: how do we define such routing rules in
>> lwIP? Hmm...
>
> I'm surprised by the question :-) In lwIP, "dev" rules don't need to be
> added, the ip4_route function "create" them dynamically using the
> interface address/netmask.

Well, yes, my bad. The question wasn't phrased well. The thing I don't
understand (because I'm not using it) probably all boils down to using
lwIP as a PPP 'proxy' to remote networks without doing NAT at the same
time (which we don't support in mainline). That's not standard routing...

>
> But thanks you for asking that because there is a huuuuuge issue about
> that with the aforementioned case. Since "dev" rules are not ordered
> because we just foreach on interfaces list until a match, we don't
> respect that smaller subnets should be checked and matched first. So if
> an IP subnet is within the IP subnet of another interface, although
> perfectly legal configuration but *bad* design, it does not work in
> lwIP. In lwIP it actually depends on the order interfaces were added :-)

Which results in me citing my mantra "lwIP does *not* support multiple
netifs in the same subnet". (And for those who ask why: most of the
time, this is just bad network desing. For the rest where it is
intended: we could make it work, but the burdon on the rest of us not
needing it has been too much, up to now.)

>
> For "via" routes, we have LWIP_HOOK_IP4_ROUTE.

Yeah, well, this is probably not used much, being a hook that people
need to implement on their own? A default implementation for a routing
algorithm woule be nice...

Regards,
Simon

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Andrew Pullin
In reply to this post by Sylvain Rochet
On 1/30/20 11:42 AM, Sylvain Rochet wrote:
> There are no ARP in PPP, but whatever, this configuration is correct.
Roger that. At the very least, ARP is enabled on the NetX equivalent of
a netif on the PPP-only host.
But it must be inactive and not accomplishing anything.
> 10.0.0.1/32 and 10.0.0.2/32 are within 10.0.0.0/16 subnet, that
> shouldn't be a problem here, this is usually a bad practice and it
> should be avoided. This is a problem for hosts on 10.0.0.0/16 that don't
> know there is a hole in the subnet, since there are only 3 hosts here,
> which is an expection rather than the rule, it is safe by luck.

This seems relatively easy to fix: I can just change the PPP-negotiated
IP's to 11.0.0.0/32, so the host1 and host2 devices are on a different
subnet on the PPP side.

As for the /32, I am unsure if I need to change that. Do I?

>> WiFi AP's:
>>      - AP only (no routing)
>>      - Connected to Host3 by Ethernet
>>      - MACs 0c:8d:db:6e:f0:03 or 0c:8d:db:6e:f0:88 (can't control
>> association)
>>
>> Host3:
>>      - Router (pfSense)
>>      - DHCP server runs here
>>      - Routes to WAN, does NAT, eth0 <-> eth1
>>      - default route: set by DHCP, toward eth1
>>      - no other route (as far as I know)
>>      - Intranet WiFi AP's are connected via switch to eth0
>>      - Intranet Ethernet devices are connected via switch to eth0
>>      - eth0 : 10.0.1.1/16 , facing intranet, MAC 00:e0:67:18:54:95
>>      - eth1 : IP from ISP, facing public IPv4 internet, MAC 00:e0:67:18:54:94
> Here is the real problem, you need a route on Host3 telling that
> 10.0.0.2/32 is behind 10.0.200.136 (or 10.0.200.113). You can also add a
> route for 10.0.0.1/32 but it's just a bonus.
>
> Without this route, Host3 don't know how to route the packet and will
> try to find an host on the 10.0.0.0/16 subnet using ARP. Then it's just
> luck, and depends on what Host3 and Host2 are doing:
>
> - If the packet is sent in a broadcast Ethernet frame by Host3 it might
> works if Host2 catches the packet and forwards it to Host1.
>
> - If the ARP request gets answered by Host2 when Host3 send their ARP
> request to find who is 10.0.0.2 on the 10.0.0.0/16 subnet, then it also
> works. This is known as ARP Proxy, lwIP does not support that.
>
> It's just luck, using lazy features of IP stack to make it works even if
> the configuration is broken should be avoided :-)
Darn. I saw some code for "UNUSED - PROXY ARP", but it was all removed
with #if 0 blocks.

>> Layout:
>>
>> Host1 <-----------------> Host2 <-------------------------> Host3
>> <---------> public internet
>>                  PPPoS                             Ethernet
>>
>> or
>>
>> Host1 <-----------------> Host2 <---------------->  WiFi AP
>> <------------------> Host3 <-------> public internet
>>                  PPPoS WiFi                              Ethernet
>>
>>
>> For the case of using the WiFi connection, there are separate WiFi AP's in
>> the way.
>> Other than that, the two cases are host2 connecting to the local subnet via
>> wlan0 or connecting via eth0.
> I guess the only reason it works with WiFi and not with Ethernet is
> because WiFi is intrinsically broadcast-based while Ethernet is switched
> (well, I hope so, unless you live in '90s :p). This is the "works
> because Host2 catches the broadcast packet" case I've said before.
Wow. What a situation ... good to know there is a rational explanation
as to why it worked with seemingly no network setup difference.

>
>> I am not manually adding any static routes in either Host1 or Host2.
>> I was worried that Host2 would not be able to accomplish the "connection
>> sharing" until I found the IP_FORWARDING option. That initially worked with
>> Host2 using WiFi without any manually added static routes.
> This is not "connection sharing", this is IP routing, not NAT !,
> "connection sharing" is usually used for hosts that do SNAT on their
> Interface facing Internet.
>
> IP routing needs a complete routing table on all hosts telling how to
> reach (i.e. which next-hop to use) every host of the whole network.
>
> Example with 2 subnets and 3 Hosts:
>
> A-eth0  --------------------- eth0-B-eth1 --------------------- eth0-C
>    192.168.0.1/24    192.168.0.2/24   172.16.1.1/24     172.16.1.2/24
>
> Host A routing table:
> 192.168.0.0/24 dev eth0
> 172.16.1.0/24 via 192.168.0.2
>
> Host B routing table:
> 192.168.0.0/24 dev eth0
> 172.16.1.0/24 dev eth1
>
> Host C routing table:
> 192.168.0.0/24 via 172.16.1.1
> 172.16.1.0/24 dev eth0
>
> dev = directly connected
> via = next-hop, gateway if you prefer
>
> What you could see is that each host have a similar routing table, just
> the next-hop changes. Default route is just a way to reduce the routing
> table size by removing all entries that have an identical next-hop.

Here is where I am starting to get worried about how I am possibly going
make this solution work.
I am trying to, in general, get my host1 device connected to the outside
world via host2's connection to host3.

The motivation for doing this via IP stacks is because my host1 (PPP
client) already has a ton of code implemented using TCP/IP; host1 is a
piece of hardware that used to have its own MAC/PHY, but is now being
repurposed. Being able to re-use all of that code without having to
remap every function at a high-level to some other mechanism (say, AT
commands, also needing host2 implementations) will be a huge boon.

My knowledge here is clearly limited, but is there a general or dynamic
way to set up the routes I would need here?
Since I want to be able to attach host2 to any existing network and have
this system work, I think this is what I would need.

I do not know how host2/B could pass the information needed to host1/A
to configure the next hop route, or "172.16.1.0/24 via 192.168.0.2" in
your example. This seems like it would require a mechanism for host2/B
to disclose to host1/A what IP address it gets (say, via DHCP) on it's
eth0 side.

Or how host3/C could be configured for routing a subnet (now
11.0.0.0/32) that is essentially "private" between host2/B and host1/A.
That is the "192.168.0.0/24 via 172.16.1.1" route in your example, or
"11.0.0.0/32 via 10.0.0.016" in my case.

It seems like what I actually do want is NAT capability on host2, so
that the host1<->host2 connection would be abstracted away.
But it looks like there are only a few few-years-old implementations of
NAT for lwip. Uh oh. I would really like to avoid trying to integrate
that in from an old branch of lwip.

Any idea if I can get this to work without needing to try to bring up NAT?

Thanks,
Andrew Pullin


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Sylvain Rochet
Hi Andrew,

On Fri, Jan 31, 2020 at 12:11:56AM -0800, Andrew Pullin wrote:

> On 1/30/20 11:42 AM, Sylvain Rochet wrote:
> >
> > 10.0.0.1/32 and 10.0.0.2/32 are within 10.0.0.0/16 subnet, that
> > shouldn't be a problem here, this is usually a bad practice and it
> > should be avoided. This is a problem for hosts on 10.0.0.0/16 that don't
> > know there is a hole in the subnet, since there are only 3 hosts here,
> > which is an expection rather than the rule, it is safe by luck.
>
> This seems relatively easy to fix: I can just change the PPP-negotiated IP's
> to 11.0.0.0/32, so the host1 and host2 devices are on a different subnet on
> the PPP side.
11.0.0.0/32 is outside RFC1918 private subnets, using it sounds like a
bad idea.


> As for the /32, I am unsure if I need to change that. Do I?

The netmask is only meaningful for layer 2 with a broadcast domain (e.g.
Ethernet), PPP is point to point, there is only one peer, netmask as no
meaning.

(To be fair that's not entirely true, it use to be used for PPP to
provide more than one IPv4 addresses to customers, but I don't think
it's still used in the wild, and it's still not a broadcast domain, it's
just a pool of addresses.)


> Darn. I saw some code for "UNUSED - PROXY ARP", but it was all removed with
> #if 0 blocks.

You are lost in the wrong place ;-), ARP Proxy support have to live in
the Ethernet ARP driver, and that support does not exist. What you saw
is just PPP support to add a Proxy ARP rule into the Ethernet driver
(basically no more than one function call :-), it is not the ARP Proxy
support.


> Here is where I am starting to get worried about how I am possibly going
> make this solution work.
> I am trying to, in general, get my host1 device connected to the outside
> world via host2's connection to host3.
>
> The motivation for doing this via IP stacks is because my host1 (PPP client)
> already has a ton of code implemented using TCP/IP; host1 is a piece of
> hardware that used to have its own MAC/PHY, but is now being repurposed.
> Being able to re-use all of that code without having to remap every function
> at a high-level to some other mechanism (say, AT commands, also needing
> host2 implementations) will be a huge boon.
>
> My knowledge here is clearly limited, but is there a general or dynamic way
> to set up the routes I would need here?
>
> Since I want to be able to attach host2 to any existing network and have
> this system work, I think this is what I would need.
>
> I do not know how host2/B could pass the information needed to host1/A to
> configure the next hop route, or "172.16.1.0/24 via 192.168.0.2" in your
> example. This seems like it would require a mechanism for host2/B to
> disclose to host1/A what IP address it gets (say, via DHCP) on it's eth0
> side.
It exists… it's called OSPF… but… erm… don't expect that to be available
behind a set-top box or whatever "gateway/router" used by ISP.

I'm running OSPF at home but I'm not the regular ISP customer :-)


> Or how host3/C could be configured for routing a subnet (now
> 11.0.0.0/32) that is essentially "private" between host2/B and
> host1/A. That is the "192.168.0.0/24 via 172.16.1.1" route in your
> example, or "11.0.0.0/32 via 10.0.0.016" in my case.
>
> It seems like what I actually do want is NAT capability on host2, so
> that the host1<->host2 connection would be abstracted away. But it
> looks like there are only a few few-years-old implementations of NAT
> for lwip. Uh oh. I would really like to avoid trying to integrate that
> in from an old branch of lwip.
>
> Any idea if I can get this to work without needing to try to bring up
> NAT?
It it were me and if I don't need Host2 to reach "outside" and Host1
software can't be changed, I would just write a big hack to forward
relevant packets (e.g. no DHCP, no ARP, IP) to the PPP link, pushing the
DHCP assigned address to the PPP peer Host1 so NAT is not necessary at
all and nothing have to be done for Host1 to send packets to "outside".

And if it were me and if I need Host2 to reach "outside" and Host1
software can't be changed, I would write a quick'n'dirty ARP Proxy
support, probably at the Ethernet low level driver. Note, in this case,
it is normal for the PPP link to be within the Host2/Host3 subnet, it is
actually mandatory. But doing so won't work if you need DHCP support
because you need DHCP server to offer two addresses (one for Host2, one
for Host1), so you need two MAC addresses and two DHCP clients running.
It's way more difficult than just hacking the forwarding and sharing the
same address.

But, above all, if it were me, I would just write a TCP or UDP proxy on
Host2. Host1 would then only establish TCP or UDP sessions to Host2 with
a home made proxy protocol (basically packets with two destination IP
fields), and Host2 would create the real session to the outside world.

Host1 ----------------- Host2 ------------ Host3 ------ Internet
192.168.0.1   192.168.0.2   10.0.200.113               192.0.2.1
<---TCP1---------------->   <---TCP2--------------------------->

TCP1:
src: 192.168.0.1
dst: 192.168.0.2
real-dst: 192.0.2.1 (in TCP payload, e.g. at the
                     beginning of the TCP session)

TCP2 (created on the fly using payload hidden real-dst):
src: 10.0.200.113
dst: 192.0.2.1


Sylvain

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Sylvain Rochet
In reply to this post by goldsimon@gmx.de
Hi Simon,

On Thu, Jan 30, 2020 at 10:11:23PM +0100, [hidden email] wrote:
>
> Well, yes, my bad. The question wasn't phrased well. The thing I don't
> understand (because I'm not using it) probably all boils down to using
> lwIP as a PPP 'proxy' to remote networks without doing NAT at the same
> time (which we don't support in mainline). That's not standard routing...

Or the contrary, that's standard IP routing, it's just totally useless
with residential ISPs because most don't allow customers to add route
entries on their gateway so it's just a flat subnet with no routing
capability. Business ISPs usually offer the option, it's very common for
business customers to have a delegated subnet that's properly routable.

Sylvain

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Andrew Pullin
In reply to this post by Sylvain Rochet
On 1/31/2020 4:48 PM, Sylvain Rochet wrote:

>> Or how host3/C could be configured for routing a subnet (now
>> 11.0.0.0/32) that is essentially "private" between host2/B and
>> host1/A. That is the "192.168.0.0/24 via 172.16.1.1" route in your
>> example, or "11.0.0.0/32 via 10.0.0.016" in my case.
>>
>> It seems like what I actually do want is NAT capability on host2, so
>> that the host1<->host2 connection would be abstracted away. But it
>> looks like there are only a few few-years-old implementations of NAT
>> for lwip. Uh oh. I would really like to avoid trying to integrate that
>> in from an old branch of lwip.
>>
>> Any idea if I can get this to work without needing to try to bring up
>> NAT?
> It it were me and if I don't need Host2 to reach "outside" and Host1
> software can't be changed, I would just write a big hack to forward
> relevant packets (e.g. no DHCP, no ARP, IP) to the PPP link, pushing the
> DHCP assigned address to the PPP peer Host1 so NAT is not necessary at
> all and nothing have to be done for Host1 to send packets to "outside".
>
> And if it were me and if I need Host2 to reach "outside" and Host1
> software can't be changed, I would write a quick'n'dirty ARP Proxy
> support, probably at the Ethernet low level driver. Note, in this case,
> it is normal for the PPP link to be within the Host2/Host3 subnet, it is
> actually mandatory. But doing so won't work if you need DHCP support
> because you need DHCP server to offer two addresses (one for Host2, one
> for Host1), so you need two MAC addresses and two DHCP clients running.
> It's way more difficult than just hacking the forwarding and sharing the
> same address.
>
> But, above all, if it were me, I would just write a TCP or UDP proxy on
> Host2. Host1 would then only establish TCP or UDP sessions to Host2 with
> a home made proxy protocol (basically packets with two destination IP
> fields), and Host2 would create the real session to the outside world.
>
> Host1 ----------------- Host2 ------------ Host3 ------ Internet
> 192.168.0.1   192.168.0.2   10.0.200.113               192.0.2.1
> <---TCP1---------------->   <---TCP2--------------------------->
>
> TCP1:
> src: 192.168.0.1
> dst: 192.168.0.2
> real-dst: 192.0.2.1 (in TCP payload, e.g. at the
>                       beginning of the TCP session)
>
> TCP2 (created on the fly using payload hidden real-dst):
> src: 10.0.200.113
> dst: 192.0.2.1

Wow, thanks for all the good ideas.
I can mostly change the code on both Host1 and Host2 here, but I am
really trying to do this with minimal changes to the upper layers,
especially on Host1. As described: end up with a transparently "working"
internet connection there.

The ARP proxy case is fascinating. Since Host1 used to be a fully
internet connected device, it has everything it would need, i.e. a DHCP
client, DNS client, etc. If I could make Host2 appear to represent two
devices with two valid (IEEE registered) MAC addresses and the network
operator was OK with two DHCP clients existing in one physical box, that
seems like a good solution.
But: modifying the low-level ethernet driver seem daunting.

The TCP proxy case also seems quite interesting, since it should be all
app layer code, and only need a TCP listener and TCP client.
Alas, we also need UDP for DNS. For TCP, it would be easy enough to
start the stream with the target IP:port and a verifier, but for UDP,
the datragram would have to be dissected, deal with the IP
pseudo-header, etc.

But here's the good news: I found a patch to add NAPT functionality to
lwip, I am sure it has been linked to somewhere in this mailing list before:
https://gist.github.com/NeoCat/da3c141813980edaa256ad351cab3a2c

Seems pretty interesting, and it seems like it is almost working, too.
This would be a great solution, since then I can run a private static IP
subnet between Host1 <-> Host2, and to the upstream router, it should
just look like a single device Host2 (other than NAT detection via TTL
or other methods).

The part that I am stuck on presently is that Host2 has no knowledge of
how to route packets from PPP out to the WAN, say, 8.8.8.8.
Between ip4_forward, IP_FORWARD, ip4_route_src, LWIP_HOOK_IP4_ROUTE,
LWIP_HOOK_IP4_ROUTE_SRC, and the ESP32 port modifications, I am not
quite sure where to make the right intervention. It is ... a lot to take in.

As far as I understand the standard usage of the stack for a single
netif, outgoing packets elide the entire forwarding/routing system
because they are specified to be transmitted on a particular netif? Is
that right?

Fundamentally, I only have eth0 and pp1  (to use the lwip netif naming
system) and anything not specifically with IP dest = eth0 or pp1 should
be sent on towards Host2's gateway (after NAT source address IP rewriting).
Is that logically complete, and if so, where should it be implemented?

Thanks,
Andrew Pullin


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

goldsimon@gmx.de
Am 11.02.2020 um 10:20 schrieb Andrew Pullin:

> [..]
>
> The part that I am stuck on presently is that Host2 has no knowledge of
> how to route packets from PPP out to the WAN, say, 8.8.8.8.
> Between ip4_forward, IP_FORWARD, ip4_route_src, LWIP_HOOK_IP4_ROUTE,
> LWIP_HOOK_IP4_ROUTE_SRC, and the ESP32 port modifications, I am not
> quite sure where to make the right intervention. It is ... a lot to take in.
>
> As far as I understand the standard usage of the stack for a single
> netif, outgoing packets elide the entire forwarding/routing system
> because they are specified to be transmitted on a particular netif? Is
> that right?
>
> Fundamentally, I only have eth0 and pp1  (to use the lwip netif naming
> system) and anything not specifically with IP dest = eth0 or pp1 should
> be sent on towards Host2's gateway (after NAT source address IP rewriting).
> Is that logically complete, and if so, where should it be implemented?
I would think you'd just set the ppp netif as default netif (so it is
just used for all outgoing trafic where no better route is found) and
keep eth0 without a gateway (which should ensure only matching subnet
packets are transmitted on this netif).

Regards,
Simon


_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: IP forwarding to/from PPP working for one netif, but not another

Sylvain Rochet
Hi Andrew,

On Tue, Feb 11, 2020 at 11:47:04AM +0100, [hidden email] wrote:

> Am 11.02.2020 um 10:20 schrieb Andrew Pullin:
> > [..]
> >
> > The part that I am stuck on presently is that Host2 has no knowledge of
> > how to route packets from PPP out to the WAN, say, 8.8.8.8.
> > Between ip4_forward, IP_FORWARD, ip4_route_src, LWIP_HOOK_IP4_ROUTE,
> > LWIP_HOOK_IP4_ROUTE_SRC, and the ESP32 port modifications, I am not
> > quite sure where to make the right intervention. It is ... a lot to take in.
> >
> > As far as I understand the standard usage of the stack for a single
> > netif, outgoing packets elide the entire forwarding/routing system
> > because they are specified to be transmitted on a particular netif? Is
> > that right?
> >
> > Fundamentally, I only have eth0 and pp1  (to use the lwip netif naming
> > system) and anything not specifically with IP dest = eth0 or pp1 should
> > be sent on towards Host2's gateway (after NAT source address IP rewriting).
> > Is that logically complete, and if so, where should it be implemented?
>
> I would think you'd just set the ppp netif as default netif (so it is
> just used for all outgoing trafic where no better route is found) and
> keep eth0 without a gateway (which should ensure only matching subnet
> packets are transmitted on this netif).
Yes, but the default netif (default route) must be set on eth0, not pp1 :)

Sylvain

_______________________________________________
lwip-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc (188 bytes) Download Attachment