This repository has been archived by the owner on Jun 11, 2021. It is now read-only.
Added new function for loading info from all interfaces #76
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi al45tair!
I would like to share this change with you, because during of dealing with my master thesis i have found a problem with getting big amount of network interfaces. Problem which i'm dealing is described here: https://bugzilla.redhat.com/show_bug.cgi?id=1582317
For big amount of network interface it takes a long time (for 5 000 NIC it's about 7 minutes) to get all needed information (like ip, netmask etc..).
I created testing scripts which i used for creating ,,dummy" NIC on my virtual machine and then i tested how long it takes to load all information about all available interfaces. So testing script for module Netifaces was like:
`
import netifaces
addrs = []
interfaces = netifaces.interfaces()
for interface in interfaces:
addrs.append(netifaces.ifaddresses(interface))
`
Results of tests are in the table below. The first column presents count of NIC available on my virtual machine. The second column presents how long it takes time to load all information about all NICs. (Values are in seconds)
Graph below shows that loading all information has exponential grow:
At the first time i thought that problem is already describe here #15, but it's not the same thing. In this pull request i have created optimization which speeds up loading names of interface about 3 time. Instead of using PySequence_Contains (..) i used dictionary, where key is name of interface and value is ,,null" then as result is returned key set from dictionary. So this was my first try.
After that, i found out where is the main problem. It's in the logic. Let's suppose we have PC with 1 000 NIC. Every NIC supports AF_INET, AF_INET6 and AF_PACKET. So library <ifaddrs.h>, respectively function getifaddrs (..) returns 3 000 records in the list (Windows library behaves same).
Now let's see on the code i have put above. This statement ,,interfaces = netifaces.interfaces()" does almost 3 000 iterations and returns list with size 1 000 (that's OK). Next statement is for loop where for each interface name is returned information about NIC and this informatio is added to list. Let's see closely on netifaces.ifaddresses(interface). Calling this function it makes again about 3 000 iteration, but only for one interface (because of finding interface with equal name on the input). It means that total count of iteration is 1 000 (NIC count) * 3 000 (calling netifaces.ifaddresses(..)). Therefore i have created new function called allifaddresses() which loads all information about all interfaces at the same time. And I added to test.py new smoke test for allifaddresses() function.
After this change the testing results are visible below:
These changes were tested on Linux/windows/MacOS and everything was OK.
Now i found out problem with appveyor. I thought the problem was in my code, but i tried to re-run appveyor test on commit https://ci.appveyor.com/project/LukasMazl/netifaces/builds/38826345 (sha 8b0d757) and it has same results (tests for 2.7, 3.4 and 3.5 do not pass)