[RFC] [RFT] Move options to config file #60

TobleMiner · 2020-04-11T14:36:25Z

This PR moves the configuration of mesh-announce to a config file. The old commandline based config interface is a hot mess.

Currently this is a breaking change. The old command line options have been removed and replaced by a config file.

@AiyionPrime @mweinelt what are your thoughts on this change? Do you agree with the overall config concept?

AiyionPrime · 2020-04-12T13:13:35Z

Have seen this, but won't look into this until thursday, if my weekplan works out.
Breaking change sounds scary :D But maybe it cleans up the code?
Will read through the commits soon; but for my own amusement. Hanover appears to be fine with the current state, as it 'works' for us.
Happy Easter and such things!

mweinelt · 2020-04-13T13:14:02Z

Thanks for making that happen, I'll also have to find time in the next few days to review and test this.

config.py

mweinelt · 2020-04-14T23:25:09Z

I like the config format and with minor nits got it up and running – but it does not seem to work properly.

Config: https://gist.github.com/mweinelt/696d5b33c08944a1f65ccb439b346cf2

It does open a socket and listen on it.

udp                UNCONN              0                    0                                                                        *:1001                   *:*            users:(("python3",pid=28615,fd=3))

But I don't see queries arriving at the process and thus no replies. The firewall rules are still in place as they were before, so they're not at fault.

# strace -p 28615                                                                                                                                                                                       [811/811]
strace: Process 28615 attached                       
restart_syscall(<... resuming interrupted read ...>) = 0
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500)   = 0 (Timeout)
poll([{fd=3, events=POLLIN}], 1, 500strace: Process 28615 detached
 <detached ...>
^C

Meanwhile queries are incoming:

# tcpdump -ni dom1-br port 1001
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on dom1-br, link-type EN10MB (Ethernet), capture size 262144 bytes
01:15:31.157162 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:16:31.157267 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:17:31.161753 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:18:31.157309 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:19:31.157313 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:20:31.157291 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:21:31.157256 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:22:31.157227 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34
01:23:31.161184 IP6 fe80::d8ff:1ff:fe00:1504.10001 > ff05::2:1001.1001: UDP, length 34

And the firewall allows that:

-A INPUT -i dom1-br -p udp -m udp --dport 1001 -j ACCEPT

Puzzling.

Misread the documentation, inheritance from the default section should now work correctly

respondd.py

mweinelt · 2020-04-15T19:56:36Z

Finally … Yanic seems to be unhappy about an empty node_id.

Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.546+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:13ff:fe00:104%dom13-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.588+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:17ff:fe00:104%dom17-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.589+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:3ff:fe00:104%dom3-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.606+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:4ff:fe00:104%dom4-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.610+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:5ff:fe00:104%dom5-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.633+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:14ff:fe00:104%dom14-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.652+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:ff:fe00:104%dom0-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.661+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:9ff:fe00:104%dom9-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.661+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:8ff:fe00:104%dom8-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.668+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:11ff:fe00:104%dom11-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.670+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:6ff:fe00:104%dom6-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.694+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:16ff:fe00:104%dom16-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.706+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:2ff:fe00:104%dom2-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.709+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:10ff:fe00:104%dom10-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.710+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:1ff:fe00:104%dom1-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.717+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:7ff:fe00:104%dom7-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.718+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:15ff:fe00:104%dom15-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"
Apr 15 21:55:30 www2.darmstadt.freifunk.net yanic[31257]: time="2020-04-15T21:55:30.719+02:00" level="warn" msg="invalid NodeID" data.address="[fe80::d8ff:12ff:fe00:104%dom12-br]:1001" data.node_id="" caller="collector.go:261 github.com/FreifunkBremen/yanic/respond.(*Collector).saveResponse"

TobleMiner · 2020-04-20T12:48:19Z

In Kiel the responses generated look perfectly normal to me

root@NightGlider:~# gluon-neighbour-info -r nodeinfo -p 1001 -d fda1:384a:74de:4242::ff00

{
  "software": {
    "fastd": {
      "enabled": true,
      "version": "v18"
    },
    "batman-adv": {
      "version": "2016.4"
    },
    "firmware": {
      "base": "Debian",
      "release": "9.11"
    }
  },
  "vpn": true,
  "node_id": "deadbeefff00",
  "hardware": {
    "nproc": 8
  },
  "network": {
    "mac": "de:ad:be:ef:ff:00",
    "mesh": {
      "bat-ffki": {
        "interfaces": {
          "tunnel": [
            "de:ad:be:ef:ff:00",
            "8a:18:67:f9:b8:08",
            "6e:f6:4e:2c:e2:6f",
            "3e:4b:3c:92:f5:20",
            "2a:f4:70:32:f9:f7",
            "92:62:15:4d:2f:8d"
          ]
        }
      }
    },
    "addresses": [
      "2a07:59c6:ec02::1",
      "2a03:2260:1017::1",
      "fe80::dcad:beff:feef:ff00",
      "fda1:384a:74de:4242::ff00"
    ]
  },
  "hostname": "vpn0"
}

I'd only expect the node id to be missing it the domain is either

not configred as a batman domain

or

the batman interface does not exist

mweinelt · 2020-04-20T13:32:50Z

# Default settings
[Defaults]
# Listen port, defaults to 1001
Port: 1001
# Default multicast listen addresses
MulticastLinkAddress: ff02::2:1001
MulticastSiteAddress: ff05::2:1001
# Default domain to use
#DefaultDomain: <domain code>
# Default domain type
DefaultDomainType: batadv

The DefaultDomainType does not take effect. Works if I set DomainType in the domain.

mweinelt · 2020-04-20T13:44:36Z

However still no domain code. Is that expected?

root@64283-darmstadtcccde:~# gluon-neighbour-info -r nodeinfo -d fd01:67c:2ed8:1001::1 | jq "."
{
  "network": {
    "addresses": [
      "fd01:67c:2ed8:1001::1",
      "fe80::d8ff:1ff:fe00:104",
      "2001:67c:2ed8:1001::1",
      "fe80::9c38:62ff:fe9d:7f7f"
    ],
    "mac": "da:ff:01:00:01:02",
    "mesh": {
      "dom1-bat": {
        "interfaces": {
          "tunnel": [
            "da:ff:01:00:01:05",
            "da:ff:01:00:01:03"
          ]
        }
      }
    }
  },
  "software": {
    "fastd": {
      "version": "v18",
      "enabled": true
    },
    "firmware": {
      "release": "10",
      "base": "Debian"
    },
    "batman-adv": {
      "version": "2019.5-1-g44fa6734"
    }
  },
  "hardware": {
    "nproc": 6
  },
  "vpn": true,
  "hostname": "gw01.darmstadt.freifunk.net",
  "node_id": "daff01000102"
}

TobleMiner · 2020-04-20T14:38:35Z

@mweinelt Ah, sorry. I changed the name of the default domain type setting to just DomainType in ce359cb. That is required for the inheritance feature of the config parser

The missing domain code is not expected, however I simply forgot to support that feature while converting to the config file based system

TobleMiner · 2020-04-20T14:45:45Z

Fixed in d3d587b

mweinelt · 2020-04-20T15:05:27Z

Yep, that makes everything work. Thanks!

herbetom · 2020-07-22T21:11:39Z

What's the holdup? In Freifunk Rhein-Neckar we started using the config file method yesterday and it seems to work.

You could of course discuss about continuing to support single domain setups via command line arguments (if there is any need), but i would tend not to do that.

AiyionPrime · 2020-07-23T12:57:19Z

Hey @herbetom ,
mind to share your config with me? Then I'll test it in Hannover as well.
Time to get this into master :)

mweinelt · 2020-07-23T13:01:57Z

# Default settings
[Defaults]
# Listen port, defaults to 1001
Port: 1001
# Default multicast listen addresses
MulticastLinkAddress: ff02::2:1001
MulticastSiteAddress: ff05::2:1001
# Default domain type
DomainType: batadv


[dom0]
# Batman interface, mandatory
BatmanInterface: dom0-bat
# Other listen interfaces
Interfaces: dom0-br, dom0-vlan, dom0-vpn-1312

[dom1]
# Batman interface, mandatory
BatmanInterface: dom1-bat
# Other listen interfaces
Interfaces: dom1-br, dom1-vlan, dom1-vpn-1312

[dom2]
# Batman interface, mandatory
BatmanInterface: dom2-bat
# Other listen interfaces
Interfaces: dom2-br, dom2-vlan, dom2-vpn-1312

[dom3]
# Batman interface, mandatory
BatmanInterface: dom3-bat
# Other listen interfaces
Interfaces: dom3-br, dom3-vlan, dom3-vpn-1312

[dom4]
# Batman interface, mandatory
BatmanInterface: dom4-bat
# Other listen interfaces
Interfaces: dom4-br, dom4-vlan, dom4-vpn-1312

[dom5]
# Batman interface, mandatory
BatmanInterface: dom5-bat
# Other listen interfaces
Interfaces: dom5-br, dom5-vlan, dom5-vpn-1312

[dom6]
# Batman interface, mandatory
BatmanInterface: dom6-bat
# Other listen interfaces
Interfaces: dom6-br, dom6-vlan, dom6-vpn-1312

[dom7]
# Batman interface, mandatory
BatmanInterface: dom7-bat
# Other listen interfaces
Interfaces: dom7-br, dom7-vlan, dom7-vpn-1312

[dom8]
# Batman interface, mandatory
BatmanInterface: dom8-bat
# Other listen interfaces
Interfaces: dom8-br, dom8-vlan, dom8-vpn-1312

[dom9]
# Batman interface, mandatory
BatmanInterface: dom9-bat
# Other listen interfaces
Interfaces: dom9-br, dom9-vlan, dom9-vpn-1312

[dom10]
# Batman interface, mandatory
BatmanInterface: dom10-bat
# Other listen interfaces
Interfaces: dom10-br, dom10-vlan, dom10-vpn-1312

[dom11]
# Batman interface, mandatory
BatmanInterface: dom11-bat
# Other listen interfaces
Interfaces: dom11-br, dom11-vlan, dom11-vpn-1312

[dom12]
# Batman interface, mandatory
BatmanInterface: dom12-bat
# Other listen interfaces
Interfaces: dom12-br, dom12-vlan, dom12-vpn-1312

[dom13]
# Batman interface, mandatory
BatmanInterface: dom13-bat
# Other listen interfaces
Interfaces: dom13-br, dom13-vlan, dom13-vpn-1312

[dom14]
# Batman interface, mandatory
BatmanInterface: dom14-bat
# Other listen interfaces
Interfaces: dom14-br, dom14-vlan, dom14-vpn-1312

[dom15]
# Batman interface, mandatory
BatmanInterface: dom15-bat
# Other listen interfaces
Interfaces: dom15-br, dom15-vlan, dom15-vpn-1312

[dom16]
# Batman interface, mandatory
BatmanInterface: dom16-bat
# Other listen interfaces
Interfaces: dom16-br, dom16-vlan, dom16-vpn-1312

[dom17]
# Batman interface, mandatory
BatmanInterface: dom17-bat
# Other listen interfaces
Interfaces: dom17-br, dom17-vlan, dom17-vpn-1312

AiyionPrime · 2020-07-23T13:07:48Z

Thanks, I'll be in freifunkh's mumble at seven, if anyone wants to join.
https://leineserver.leinelab.org/
Default config if you join using the mumbleclient.

AiyionPrime · 2020-07-23T20:34:36Z

This branch looks promising.
Argc is drastically reduced, as intended;
The configfile was a breeze to setup.

As soon as this branch gets merged,
I'll put the altered configuration into our ansible repo.

I'd vote for merge.

btw, is there an option to override the hostname?

herbetom · 2020-08-05T18:55:18Z

btw, is there an option to override the hostname?

I don't think so, but with the config file it should be easilier possible to add such an option, since their is to need to add another argc. 😄

TobleMiner · 2020-09-07T21:42:01Z

Merged

AiyionPrime · 2020-09-07T23:38:45Z

I just announced in hanover, that we'll upgrade as soon as possible an then reevaluate, whether this accidentally resolved #52 or parts of #57.

they implemented a long awaited PR, where they'd change the config from arguments to one configfile. Look here for further reference: ffnord/mesh-announce#60

TobleMiner added 11 commits April 10, 2020 16:59

Replace argument based config system by config file

065fa9c

Replace VPNInterface by Interfaces option

dbdbca1

Remove unused utility functions

e3c7785

Add some more function and class level documentation

d0a6f98

show default config file location

2e24b20

Update README for new config system

5b91f24

Support default domain

cbbbb9c

Add simple domain type

08d7336

Split commandline section

579c172

Enhance example config documentation

f7f3e18

Import domain options

0bbdd2d

mweinelt reviewed Apr 14, 2020

View reviewed changes

config.py Outdated Show resolved Hide resolved

mweinelt reviewed Apr 14, 2020

View reviewed changes

config.py Outdated Show resolved Hide resolved

TobleMiner added 4 commits April 15, 2020 11:27

Don't shadow type() function

3d462c3

Fix domain type fallback

6781afa

Use configparser correctly

ce359cb

Misread the documentation, inheritance from the default section should now work correctly

Update config example

4e0d1bd

mweinelt reviewed Apr 15, 2020

View reviewed changes

respondd.py Show resolved Hide resolved

State format of interface specification

0c80ffd

Add domain_code to provider env

d3d587b

TobleMiner merged commit 2736f7b into ffnord:master Sep 7, 2020

This was referenced Feb 14, 2021

Update mesh announce freifunkh/ansible#148

Merged

feature: option to override the hostname #65

Closed

herbetom added a commit to Freifunk-Rhein-Neckar/mesh-announce that referenced this pull request Feb 18, 2021

delete domainfile.json.example: not neded after ffnord#60 was merged

27d1a9f

herbetom mentioned this pull request Feb 18, 2021

delete domainfile.json.example: not neded after #60 was merged #72

Merged

TobleMiner pushed a commit that referenced this pull request Mar 21, 2021

delete domainfile.json.example: not neded after #60 was merged

4cd6cd3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] [RFT] Move options to config file #60

[RFC] [RFT] Move options to config file #60

TobleMiner commented Apr 11, 2020

AiyionPrime commented Apr 12, 2020 •

edited

Loading

mweinelt commented Apr 13, 2020

mweinelt commented Apr 14, 2020 •

edited

Loading

mweinelt commented Apr 15, 2020

TobleMiner commented Apr 20, 2020 •

edited

Loading

mweinelt commented Apr 20, 2020 •

edited

Loading

mweinelt commented Apr 20, 2020

TobleMiner commented Apr 20, 2020

TobleMiner commented Apr 20, 2020

mweinelt commented Apr 20, 2020

herbetom commented Jul 22, 2020 •

edited

Loading

AiyionPrime commented Jul 23, 2020

mweinelt commented Jul 23, 2020

AiyionPrime commented Jul 23, 2020

AiyionPrime commented Jul 23, 2020 •

edited

Loading

herbetom commented Aug 5, 2020

TobleMiner commented Sep 7, 2020

AiyionPrime commented Sep 7, 2020 •

edited

Loading

[RFC] [RFT] Move options to config file #60

[RFC] [RFT] Move options to config file #60

Conversation

TobleMiner commented Apr 11, 2020

AiyionPrime commented Apr 12, 2020 • edited Loading

mweinelt commented Apr 13, 2020

mweinelt commented Apr 14, 2020 • edited Loading

mweinelt commented Apr 15, 2020

TobleMiner commented Apr 20, 2020 • edited Loading

mweinelt commented Apr 20, 2020 • edited Loading

mweinelt commented Apr 20, 2020

TobleMiner commented Apr 20, 2020

TobleMiner commented Apr 20, 2020

mweinelt commented Apr 20, 2020

herbetom commented Jul 22, 2020 • edited Loading

AiyionPrime commented Jul 23, 2020

mweinelt commented Jul 23, 2020

AiyionPrime commented Jul 23, 2020

AiyionPrime commented Jul 23, 2020 • edited Loading

herbetom commented Aug 5, 2020

TobleMiner commented Sep 7, 2020

AiyionPrime commented Sep 7, 2020 • edited Loading

AiyionPrime commented Apr 12, 2020 •

edited

Loading

mweinelt commented Apr 14, 2020 •

edited

Loading

TobleMiner commented Apr 20, 2020 •

edited

Loading

mweinelt commented Apr 20, 2020 •

edited

Loading

herbetom commented Jul 22, 2020 •

edited

Loading

AiyionPrime commented Jul 23, 2020 •

edited

Loading

AiyionPrime commented Sep 7, 2020 •

edited

Loading