Skip to content
This repository has been archived by the owner on Jun 11, 2021. It is now read-only.

Added new function for loading info from all interfaces #76

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

LukasMazl
Copy link

Hi al45tair!

I would like to share this change with you, because during of dealing with my master thesis i have found a problem with getting big amount of network interfaces. Problem which i'm dealing is described here: https://bugzilla.redhat.com/show_bug.cgi?id=1582317
For big amount of network interface it takes a long time (for 5 000 NIC it's about 7 minutes) to get all needed information (like ip, netmask etc..).

I created testing scripts which i used for creating ,,dummy" NIC on my virtual machine and then i tested how long it takes to load all information about all available interfaces. So testing script for module Netifaces was like:
`
import netifaces

addrs = []
interfaces = netifaces.interfaces()
for interface in interfaces:
addrs.append(netifaces.ifaddresses(interface))
`
Results of tests are in the table below. The first column presents count of NIC available on my virtual machine. The second column presents how long it takes time to load all information about all NICs. (Values are in seconds)

Capture

Graph below shows that loading all information has exponential grow:
Speed test

At the first time i thought that problem is already describe here #15, but it's not the same thing. In this pull request i have created optimization which speeds up loading names of interface about 3 time. Instead of using PySequence_Contains (..) i used dictionary, where key is name of interface and value is ,,null" then as result is returned key set from dictionary. So this was my first try.

After that, i found out where is the main problem. It's in the logic. Let's suppose we have PC with 1 000 NIC. Every NIC supports AF_INET, AF_INET6 and AF_PACKET. So library <ifaddrs.h>, respectively function getifaddrs (..) returns 3 000 records in the list (Windows library behaves same).
Now let's see on the code i have put above. This statement ,,interfaces = netifaces.interfaces()" does almost 3 000 iterations and returns list with size 1 000 (that's OK). Next statement is for loop where for each interface name is returned information about NIC and this informatio is added to list. Let's see closely on netifaces.ifaddresses(interface). Calling this function it makes again about 3 000 iteration, but only for one interface (because of finding interface with equal name on the input). It means that total count of iteration is 1 000 (NIC count) * 3 000 (calling netifaces.ifaddresses(..)). Therefore i have created new function called allifaddresses() which loads all information about all interfaces at the same time. And I added to test.py new smoke test for allifaddresses() function.

After this change the testing results are visible below:

repair

These changes were tested on Linux/windows/MacOS and everything was OK.

Now i found out problem with appveyor. I thought the problem was in my code, but i tried to re-run appveyor test on commit https://ci.appveyor.com/project/LukasMazl/netifaces/builds/38826345 (sha 8b0d757) and it has same results (tests for 2.7, 3.4 and 3.5 do not pass)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant