Netkit-JH 2.0.0

This is a full refactor of the Netkit-JH codebase, which focuses on cleaning up the original code from Netkit and Netkit-NG. Some functionality has been added or modified, and some features have been removed if deprecated or redundant; backwards compatibility largely still exists for regular usage.

It is highly recommended to read the changelog below!

Due to the scale of this update, there may be more bugs compared to other releases. Every commonly used option and configuration has been tested extensively to the best of the maintainers' abilities, but there may be bugs in less commonly used options or if the user is doing something very funky (chances are the bug you found has laid dormant for years or even decades!). The user is recommended to report any bugs via GitHub or the Discord, where they will get fixed ASAP either as hotfixes or new versions.

Notable features include:

• Whitespace in arguments or paths does not completely break operation
• netkit.conf.default stores default configuration variables that can be overridden with netkit.conf
• The kitty-tabs terminal emulator option has been fixed and upgraded, and is now called kitty-tab
• Multiple tap interfaces can coexist and are user-specific (they are now specified with DOMAIN,TAP-ADDR,GUEST-ADDR, where DOMAIN doesn't have to be "tap")
• There is now Bash tab completion
• (For development) ShellCheck directives have been added and copyright notices now feature the Netkit-JH development team

The full changelog is as follows:

User Experience Changelog

Added

• Command tab-completion (in Bash) including machine name and collision domain completion
• --mount option to vstart - mount any host folder inside a machine
• Ability to use a _test/shared.test script to run the same script for all hosts with ltest. Script output is stored in _test/results/$hostname.shared
• Multiple taps can be created for machines to have their own unique interfaces (they can still share if the same collision domain is used)
• Support for whitespace (and some other special characters) in arguments
• Support for whitespace in the lab directory path
• --clean-directories option to vclean to wipe $HOME/.netkit/ (ignoring netkit.conf), $MCONSOLE_DIR/, and $HUB_SOCKET_DIR/ (now included in --clean-all)
• --keep-fs option to vcrash (default operation now removes the filesystem like lcrash, see Modified)

Other Added

• -v|--verbose option to lcrash, vclean, vconf, vcrash, and vdump
• -f|--force option to vclean for usage alongside -H|--remove-hubs to remove hubs even if they are being used by running machines
• --delay option to ltest to specify how long to wait for the lab to settle before running test scripts
• --show-boot-log option to vstart to display the boot log (replaced the -v|--verbose option, see Other Modified)
• --version option to install-netkit-jh.sh
• Ability to affect a specific user for -T|--remove-tunnels in vclean
• bash, binutils, coreutils, iproute2, lsof, and util-linux as dependencies
• Virtual hub logging can be disabled by setting the HUB_LOG variable to an empty string

Modified

• Manual pages are completely up-to-date and (hopefully!) sufficiently detailed
• Netkit's default configuration is stored in netkit.conf.default. This should not be modified and can be overridden with netkit.conf
• Tap interfaces are specified with DOMAIN,TAP-ADDR,GUEST-ADDR (DOMAIN does not have to be tap now, it should be a regular collision domain name). See Added for information on multiple tap interface specifications
• The kitty-tabs terminal emulator option is now called kitty-tab
• vcommand usage signature is now vcommand [OPTION]... MACHINE [COMMAND]
• vconnect usage signature is now vconnect [OPTION]... MACHINE
• machines lab.conf variable is now LAB_MACHINES
• Minimum Bash version (4.4) is enforced in the 1_check_shell.sh configuration check script
• LAB_MACHINES variable is comma-delimited (orig. whitespace-delimited)
• vclean will kill tmux sessions alongside their machines
• ltest user defined scripts should be stored in _test/scripts/
• ltest script results are placed in _test/results/
• 'tap' interfaces are created under a /24 subnet now (not /8). This allows more 'tap' interfaces on the host to occupy private address spaces
• vcrash now deletes filesystems by default, like lcrash. The --remove-fs option has been replaced by --keep-fs (see Added and Removed)
• The Netkit-JH kernel has been updated from Linux version 5.14.9 to 5.19.11
• The wsl option to vstart --xterm has been renamed to conhost and now explicitly calls conhost.exe, as opposed to Windows' default terminal emulator

Other Modified

• vconf manual entry is called vconf now (used to be vconfig)
• -v|--verbose in vstart now only displays machine information and ran commands. Boot log verbosity is now controlled with --show-boot-log, see Other Added
• Rename --kill to --just-kill in vcrash and lcrash
• -p|--print option has been changed to -n|--dry-run|--just-print|--recon for vconf, vpackage, and vstart. This follows what the Make utility uses
• --clean-all in vclean now removes all items inside Netkit directories ($HOME/.netkit/, $MCONSOLE_DIR/, and $HUB_SOCKET_DIR/)
• --clean-all in vclean now affects machines owned by all users
• Machine names must conform to the Debian standard (which conforms to the RFC standard too). See hostname_regex in core/bin/script_utils for more guidance
• Collision domain names must conform to a regular expression and length-check. See collision_domain_name_regex in core/bin/script_utils for more guidance
• Better output when multiple labs are specified in linfo -m
• tmux sessions are now named netkit-vhost (orig. netkit-vm)
• Tap interfaces are named nk_$interface_sha256, where interface_sha256 is an 12-character (truncated) SHA-256 digest of the collision domain name SHA-256 appended to the owner's username's SHA-256 for uniqueness and avoiding character set issues
• iptables forwarding rule uses the new interface specification to match the input interface (nk_+)
• Virtual network hub sockets are named with the username's and domain's SHA-256 digests instead of plaintext to avoid issues with invalid characters. Tap sockets are identfied in the filename
• lstart now ignores machines unable to start, as opposed to fatally exiting
• lab.conf files will get checked for trailing whitespace
• /etc/vhostconfigured (file indicating machine has been booted before) is now at /etc/netkit/.vhostconfigured
• The netkit-phase1 and netkit-phase2 services now run explicitly from /root (which was always implicit behaviour)
• vdump outputs --verbose messages to standard error instead of standard output to avoid mixing with the packet data
• linfo -m will now escape DOT and XML metacharacters in hostnames, collision domain names, and other identifiers printed in the map
• Error and warning messages are now coloured red and purple respectively

Removed

• --machine and --command switches in vcommand (see Modified)
• --machine switch in vconnect (see Modified)
• ltest signature comparison, and the -R|--rebuild-signature and --verify options
• -S|--script-mode option from ltest
• USE_SUDO configuration directive (default is to use sudo now, su -c has been removed)
• --quiet option for lcrash, lhalt, vclean, vcrash, and vhalt (-q|--quick still exists)
• -q|--quiet option from vconf and vstart
• konsole-tab argument to the --xterm option (and the ktabstart script)
• bspwm compatibility patch files. They fail with the radically changed target files and fix a general issue other users have identified but only for bspwm - it applies to tmux, Windows Terminal, and other terminal emulators and window managers
• Dependency on the net-tools (ifconfig) package for manage_tuntap in favour of iproute2
• --remove-fs option from vcrash (now the default operation like lcrash; replaced by --keep-fs, see Modified)

Other Removed

• Port helper compatibility
• -w|--hostwd option from vstart (it did not provide any purpose)
• lrestart symbolic link (pointed to lstart and provided no default or additional functionality)
• --fix option from check_configuration.sh
• MCONSOLE_DIR environment variable detection from uml_mconsole
• Support for netkit.conf overriding with the NETKIT_FILESYSTEM, NETKIT_MEMORY, NETKIT_KERNEL, and NETKIT_TERM environment variables
• 01-check_path.sh setup script (whitespace in paths is safely handled now)
• 06-check_aux_tools.sh and 10-check_dumpers.sh setup scripts

Development Changelog

Added

• Netkit's default configuration is stored in netkit.conf.default. This should not be modified and can be overridden with netkit.conf. This allows for smoother updates of configuration variables and their default values, since the default file can be entirely replaced with the new version (as opposed to using a janky sed script)
• ShellCheck directives for effective linting
• .gitignore now ignores compilation, test, and development artefacts
• Copyright notices for the Netkit-JH development team (alongside the original Netkit and Netkit-NG notices)
• Collision domain argument to manage_tuntap
• vcommon source script to contain functions common to the vcommands (analogous to lcommon)

Modifed

• All scripts are interpreted by Bash now (minimum required version 4.4, see Modified)
• Script order:
  1. /usr/bin/env bash shebang
  2. Copyright notice
  3. Script description
  4. `usage...

Netkit 2.0.0-beta

This is a full refactor of the Netkit-JH codebase, which focuses on cleaning up the original code from Netkit and Netkit-NG. Some functionality has been added or modified, and some features have been removed if deprecated or redundant; backwards compatibility largely still exists for regular usage. It is highly recommended to read the changelog below!

The man pages and Makefiles have not been refactored yet.

The refactor has been tested manually to the best of my abilities, but understandably there may be bugs in less commonly used options or if the user is doing something very funky. The user is recommended to test this out and report any bugs via GitHub or the Discord, where they will get fixed ASAP in preparation for the stable 2.0.0 release.

Notable features include:

• Whitespace in arguments or paths does not completely break operation
• netkit.conf.default stores default configuration variables that can be overridden with netkit.conf
• The kitty-tabs terminal emulator option has been fixed and upgraded, and is now called kitty-tab
• Multiple tap interfaces can coexist and are user-specific (they are now specified with DOMAIN,TAP-ADDR,GUEST-ADDR, where DOMAIN doesn't have to be "tap")
• There is now Bash tab completion
• (For development) ShellCheck directives have been added and copyright notices now feature the Netkit-JH development team

The full changelog is as follows:

User Experience Changelog

Added

• Command tab-completion (in Bash) including machine name and collision domain completion
• --mount option to vstart - mount any host folder inside a machine
• Ability to use a _test/shared.test script to run the same script for all hosts with ltest. Script output is stored in _test/results/$hostname.shared
• Multiple taps can be created for machines to have their own unique interfaces (they can still share if the same collision domain is used)
• Support for whitespace (and some other special characters) in arguments
• Support for whitespace in the lab directory path
• --clean-directories option to vclean to wipe $HOME/.netkit/ (ignoring netkit.conf), $MCONSOLE_DIR/, and $HUB_SOCKET_DIR/ (now included in --clean-all)

Other Added

• -v|--verbose option to lcrash, vclean, vconf, vcrash, and vdump
• -f|--force option to vclean for usage alongside -H|--remove-hubs to remove hubs even if they are being used by running machines
• --delay option to ltest to specify how long to wait for the lab to settle before running test scripts
• --show-boot-log option to vstart to display the boot log (replaced the -v|--verbose option, see Other Modified)
• --version option to install-netkit-jh.sh
• Ability to affect a specific user for -T|--remove-tunnels in vclean
• bash, binutils, coreutils, iproute2, lsof, and util-linux as dependencies

Modified

• Netkit's default configuration is stored in netkit.conf.default. This should not be modified and can be overridden with netkit.conf
• Tap interfaces are specified with DOMAIN,TAP-ADDR,GUEST-ADDR (DOMAIN does not have to be tap now, it should be a regular collision domain name). See Added for information on multiple tap interface specifications
• The kitty-tabs terminal emulator option is now called kitty-tab
• vcommand usage signature is now vcommand [OPTION]... MACHINE [COMMAND]
• vconnect usage signature is now vconnect [OPTION]... MACHINE
• machines lab.conf variable is now LAB_MACHINES
• Minimum Bash version (4.4) is enforced in the 1_check_shell.sh configuration check script
• LAB_MACHINES variable is comma-delimited (orig. whitespace-delimited)
• ltest user defined scripts should be stored in _test/scripts/
• ltest script results are placed in _test/results/
• 'tap' interfaces are created under a /24 subnet now (not /8). This allows more 'tap' interfaces on the host to occupy private address spaces

Other Modified

• vconf manual entry is called vconf now (used to be vconfig)
• -v|--verbose in vstart now only displays machine information and ran commands. Boot log verbosity is now controlled with --show-boot-log, see Other Added
• Rename --kill to --just-kill in vcrash and lcrash
• -p|--print option has been changed to -n|--dry-run|--just-print|--recon for vconf, vpackage, and vstart. This follows what the Make utility uses
• --clean-all in vclean now removes all items inside Netkit directories ($HOME/.netkit/, $MCONSOLE_DIR/, and $HUB_SOCKET_DIR/)
• --clean-all in vclean now affects machines owned by all users
• Machine names must conform to the Debian standard (which conforms to the RFC standard too). See hostname_regex in core/bin/script_utils for more guidance
• Collision domain names must conform to a regular expression and length-check. See collision_domain_name_regex in core/bin/script_utils for more guidance
• Better output when multiple labs are specified in linfo -m
• tmux sessions are now named netkit-vhost (orig. netkit-vm)
• Tap interfaces are named nk_$interface_sha256, where interface_sha256 is an 12-character (truncated) SHA-256 digest of the collision domain name SHA-256 appended to the owner's username's SHA-256 for uniqueness and avoiding character set issues
• iptables forwarding rule uses the new interface specification to match the input interface (nk_+)
• Virtual network hub sockets are named with the username's and domain's SHA-256 digests instead of plaintext to avoid issues with invalid characters. Tap sockets are identfied in the filename
• /etc/vhostconfigured (file indicating machine has been booted before) is now at /etc/netkit/.vhostconfigured

Removed

• --machine and --command switches in vcommand (see Modified)
• --machine switch in vconnect (see Modified)
• ltest signature comparison, and the -R|--rebuild-signature and --verify options
• -S|--script-mode option from ltest
• USE_SUDO configuration directive (default is to use sudo now, su -c has been removed)
• --quiet option for lcrash, lhalt, vclean, vcrash, and vhalt (-q|--quick still exists)
• -q|--quiet option from vconf and vstart
• konsole-tab argument to the --xterm option (and the ktabstart script)
• bspwm compatibility patch files. They fail with the radically changed target files and fix a general issue other users have identified but only for bspwm - it applies to tmux, Windows Terminal, and other terminal emulators and window managers
• Dependency on the net-tools (ifconfig) package for manage_tuntap in favour of iproute2

Other Removed

• Port helper compatibility
• -w|--hostwd option from vstart (it did not provide any purpose)
• lrestart symbolic link (pointed to lstart and provided no default or additional functionality)
• --fix option from check_configuration.sh
• MCONSOLE_DIR environment variable detection from uml_mconsole
• Support for netkit.conf overriding with the NETKIT_FILESYSTEM, NETKIT_MEMORY, NETKIT_KERNEL, and NETKIT_TERM environment variables
• 01-check_path.sh setup script (whitespace in paths is safely handled now)
• 06-check_aux_tools.sh and 10-check_dumpers.sh setup scripts

Development Changelog

Added

• Netkit's default configuration is stored in netkit.conf.default. This should not be modified and can be overridden with netkit.conf. This allows for smoother updates of configuration variables and their default values, since the default file can be entirely replaced with the new version (as opposed to using a janky sed script)
• ShellCheck directives for effective linting
• .gitignore now ignores compilation, test, and development artefacts
• Copyright notices for the Netkit-JH development team (alongside the original Netkit and Netkit-NG notices)
• Collision domain argument to manage_tuntap
• vcommon source script to contain functions common to the vcommands (analogous to lcommon)

Modifed

• All scripts are interpreted by Bash now (minimum required version 4.4, see Modified)
• Script order:
  1. /usr/bin/env bash shebang
  2. Copyright notice
  3. Script description
  4. usage_line & usage
  5. Other function definitions in order of usage
  6. script_utils and (optional) lcommon/vcommon source
  7. getopt argument parsing
  8. Non-option argument parsing
  9. Remainder of script
• hostname parameter in kernel_cmd (replaced name and title)
• Renamed check_configuration.d/ scripts

Removed

• name and title parameters in kernel_cmd
• /etc/netkit/netkit-welcome script (was disabled anyway)
• check.sh filesystem check script

Bug Fixes

• kitty-tab terminal emulator option has been upgraded and now works
• uml_switch logging now works
• ltest automated lab testing now works (added /etc/netkit/netkit-test-phase)

Known Issues

• The man pages have not been updated yet
• Removing an interface with vconf crashes the machine
• Whitespace in the VM_MODEL_FS (-m|--model-fs) and VM_KERNEL (--kernel) configuration variables, HOME environment variable, or the argument to --mount, --hostlab, and --exec may break when ran with Windows Terminal due to escaped quotes not being honoured by the extra layer of shell interpretation.
• A terminating quotation mark in the VM_MODEL_FS (-m|--model-fs) configuration variable will be removed by the get_vhost_info_by_pid function if the instance is running in Windows Terminal. This is significant in vcrash's and vhalt's --remove-fs option - the disk file will not be removed (and a file with the same name sans quote mark will be, if exists).

Netkit 1.1.4

New:

• Tmux is now it's own con0 option
• Updated to Linux kernel 5.4.19.
• Updated to stable release Debian Bullseye.
• Added ettercap.
• Add Windows Terminal support for WSL hosts.
• Added vpackage script to install new packages onto the fs.

Bug fixes:

• Fixed filesystem permissions of copied files in build process.
• Install script now stops on an error, rather than silently erroring.
• Disabled xl2tpd service.
• Various fixes for invalid/legacy Bash scripts.

Removed:

• Removed rng-tools + systemd entropy patch.
• Removed proftpd due to installation issues.

Other:

• Localised UML utilities
• Increased default MAX_SIMULTANEOUS_VMS variable as there's no disadvantage to having it increased.

Thank you to everyone who have helped add various features within this release.

The install script and release.sha256 files have been updated since initial release to fix installation issues.

Netkit 1.1.3

Bug fixes:

• Fixed con0 = this / con0 = none
• Root is no longer using a locked password and instead has an empty password

New:

• Added sudo package

This update is smaller than usual and if you have no problems with 1.1.2, it's not an essential update :-)

Netkit 1.1.2

Bug fixes:

• .startup files are now sent to /bin/bash rather than /bin/sh
• exim4 SMTP server is now longer started by default
• Default terminal is set back to xterm
• Installation script correctly uses check_configuration.sh

Additions:

• Installation script also supports alternative shells such as ZSH
• linfo can now handle split labs
• Better support for running Netkit in WSL

Thank you to all the contributors for this release!

Netkit v1.1.1

Bug fixes:

• Remove hardcoded netkit installation directory on update checker
• Add curl to install packages
• Fix broken configuration upgrade
• Updated Docker file to improve building edge cases and included docker-script.sh to assist with running it

Netkit 1.1.0

Compiling:

• Cleaned up Makefiles for all aspects.
  • make install now uses compiled modules to install, rather than creating archives and extracting them needlessly.
  • make release now creates release files with associated archives and hash file.
  • Makefiles (for the most part) no longer run when nothing has changed.
• Removed uml-utilities from our repo (for the most part, it is still required for uml_dump)
• Cleaned up compile files in general.
• Removed Debian from the build process, you can now build natively on any machine with debootstrap.

New:

• Linux kernel 5.10.4
• Gnome-terminal now works (after 10 years!) Use change_terminal.sh to change to gnome.
• TMUX term type allows you to background labs and connect to them one at a time. Useful for large labs where you only want to view 1 or 2 machines at a time.
  • vconnect provides an interactive shell to the machine
  • vcommand runs a command inside a backgrounded machine and outputs the result
• linfo -m now produces a PNG rather than a PostScript file and has a cleaned up style.
• Removal of uml tools within the repo (note, this means you will need to install uml-utilities!)
• Improved install script will copy your old configurations over when installing + general improvements to the install script.
• Updated check_configuration.sh with new terminals.
• Proper testing suite allows us to verify our new releases work quickly.
• Updated man pages with new maintainers.
• Added support for ZSH.
• Added auto-update checker to alert you of new releases of Netkit-JH.
• Added debug mode to debug Netkit via GDB.

Bug fixes:

• Fixed vclean --clean-all displaying an error.
• Fixed unable to copy files with spaces into lab machines.
• Improved iptables rules to stop interference with other MASQUERADE rules.
• Fix /etc/resolv.conf not existing.
• Added warning about lack of graphics drivers for Alacritty.

Most importantly, we've migrated the project to it's own Github account. With this, we've detached from the netkit-ng fork and also created a proper website for the Netkit-JH project. Check it out for guides on installing and developing: https://netkit-jh.github.io/.

Thank you to everyone who has helped out on this release, it has been great seeing so many issues and pull requests being submitted.

Netkit 1.0.0

Compiling:

• Improved build process for the filesystem (decreasing build times from 40 minutes to 10 minutes!) which will make updating and maintaining the filesystem easier.
• Improved build process for all modules
• Moved core tools into the build repo
• Updated UML tools to 64 bit (hoorah!)
• Use the UK debian repos to make downloading packages that little bit faster
• Added dockerfile to compile using Docker - thanks @b177y

New in this release:

• LFTP & at packages
• Added support for Kitty & Alacritty terminal emulators
• New install script which asks what terminal emulator you wish to use (and handles installation of it!)
• Added message about 'lclean' when labs don't start properly

Bug fixes:

• Removed mmapper_init failed error message
• Removed failed to load autofs4 module error message
• Set timeout of startup script service to 5 minutes
• vstart --version now lists the correct kernel/fs version

Removed:

• Netkit welcome message has been removed (was broken before due to a bad systemd service config)

Massive thanks to @okman12 and @b177y for assisting me in improving this version :-)

Most importantly, we're now at v1.0.0 of our new Netkit :) Our improvements to the building process (removal of rootstrap, combining of the two repos) and our upgrade to Debian Bullseye + Linux Kernel 5.9 justify a big version jump in my opinion. So welcome to v1.0.0, the best update we've had so far :-)

Coming in the future:

• Removing net-tools dependancy from core tools
• Exploring the concept of different distros for filesystems
• Making it more stable :-)

Netkit 0.1.8

Bug fixes:

• Fixed Wireguard not being built into the kernel
• lclean now removes left over mconsole files (solves the issue of machines kernel panicing when closed incorrectly)
• Fixed bug where writing lab files to a symlinked directory (e.g. /lib) would cause the machine to not start up correctly
• Fixed netkit-welcome not being ran
• Fixed MANPATH export wiping previous MANPATH exports (in install script)

Not in this release:

• Updating debian repos to use UK versions

SHA256sums:
143e66edd6b972679b93a8b6933fbfb879c4b7173e7f858539c0c1c58c079bcd netkit-core-0.1.8.tar.bz2
9252710b181578175aa552c0fe48811299010bae3c91a34ef3cf43d9d1c8f083 netkit-fs-0.1.8.tar.bz2
bf1a269c5c3512fcf7fbddc0913c884d7445143e23ad76ce98221c02c9ac9326 netkit-kernel-0.1.8.tar.bz2

Netkit 0.1.7

Bug fixes:

• Released kernel now using non-random, actually unique MAC addresses (whoops!)
• Updated permissions on systemd services

Updates:

• Added traceroute package

Other:

• Cleaned up build scripts to make releasing new versions easier

SHA256sums:
6dc218491c25b55be0e19b179fb630d7167066707f43a0db72123ebd2c494c17 netkit-core-0.1.7.tar.bz2
8510f23688271d77251a1ad74fd1b464f1bbe66a134394e5e82410dd188c8c12 netkit-fs-0.1.7.tar.bz2
f3c4c8112829303637d9446e6042ffc1d5c446faac3765c67ef5f45516b0468e netkit-kernel-0.1.7.tar.bz2