From 5b0480312b7751e597bf78e15a6be3e93dbb8290 Mon Sep 17 00:00:00 2001 From: Serge Aleynikov Date: Tue, 16 Aug 2016 00:03:11 -0400 Subject: [PATCH 1/2] Add Alternative Packet Block This modification is based on the proposal submitted to the pcap-ng-formati mailing list. It introduces two new features: - Alternative Packet Block - Simple Option Type --- draft-tuexen-opsawg-pcapng.xml | 155 ++++++++++++++++++++++++++++++++- 1 file changed, 152 insertions(+), 3 deletions(-) diff --git a/draft-tuexen-opsawg-pcapng.xml b/draft-tuexen-opsawg-pcapng.xml index 60fa4e8..f591abe 100644 --- a/draft-tuexen-opsawg-pcapng.xml +++ b/draft-tuexen-opsawg-pcapng.xml @@ -632,6 +632,40 @@ Section Header +
+ + Simple Options are used for represending integer values that fit + into 24 bits of space. + +
+ + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Option Code | Option Value | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +
+ + A Simple Option has the following fields: + + + Simple Option Code: The code number for the Simple Option, which + has the upper three bits set to '001'. Given the total size of the + option code equal to 8 bits, there are at most 32 distinct simple + option codes available for each block type. + + + Option Value: An integer value stored in 24bits. + + + + + See for examples of Simple Options + usage. + +
+
@@ -1903,11 +1937,126 @@ Section Header
-
+
+ + Alternative Packet Block offers an intermediate layout between + the Simple Packet Block and the Enhanced Packet Block. Just like the + Enhanced Packet Block it stores a timestamp of each packet, but + all other fields in the fixed part of the Alternative Packet + Block's body are made optional and can be stored in the Options + section. The packet's body of this block MAY be compressed. + + +
+ + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +---------------------------------------------------------------+ + 0 | Block Type = 0x00000010 | + +---------------------------------------------------------------+ + 4 | Block Total Length | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 8 | Timestamp (High) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +12 | Timestamp (Low) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +16 / / + / Options (variable) / + / / + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + / / + / Packet Data / + / variable length, padded to 32 bits / + / / + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Block Total Length | + +---------------------------------------------------------------+ + +
+ + The permissible options listed in the table below are of + Simple Option type (see ) + + + + Name + Code + Length + Multiple allowed? + + apb_opt_size + 0x2001 + - + no + + apb_orig_len + 0x2002 + - + no + + apb_iface_id + 0x2003 + - + no + + apb_flags + 0x2004 + - + no + + + + + + Total byte size of options. If this option is specified, then + the opt_endofopt option is OPTIONAL. The value of apb_opt_size + is the number of octets comprising the entire Options section in + this block. If present, this option MUST be the first one in the + Options data. + + Packet's + Original Length. Actual length of the packet when it was + transmitted on the network. It can be different from captured + packet length, which is the min(snapshot length, bytes left in + the block), if the packet has been truncated by the capture + process. + + It + specifies the interface this packet comes from; the correct + interface will be the one whose Interface Description Block + (within the current Section of the file) is identified by + the same value (see ) of this + option. The interface ID MUST be valid, which means that an + matching interface description block MUST exist. If this option + is not defined, interface ID defaults to 0. + + Alternative + Packet Block Flags indicate characteristics of the Packet Data. + See for permissible bit settings. + + - Can some other packet blocks (besides the ones described in the - previous paragraphs) be useful? + + Bit Number + Description + 0-1 + Inbound / Outbound packet (00 = information not + available, 01 = inbound, 10 = outbound) + + 2-4 + Reception type (000 = not specified, 001 = unicast, + 010 = multicast, 011 = broadcast, 100 = promiscuous). + + 5-12 + Compression type. When specified, the Packet Data + content is compressed. Permissible types: + 0 (uncompressed), 1 (lzw), 2 (gzip), 3 (bzip2), 4 (zip), + 5 (7z), 6 (lzo), 7 (ucl), 8 (snappy), other??? + + 13-23 + Reserved +
From e6bd08284245c6a3864877bbae8d9e0c0a5af48c Mon Sep 17 00:00:00 2001 From: Serge Aleynikov Date: Tue, 16 Aug 2016 10:06:51 -0400 Subject: [PATCH 2/2] Make timestamp optional This change makes the timestmap optional and also moves it past the Options data. This way if the packet is compressed, the compression would also include the options, as well as the timestamp. --- draft-tuexen-opsawg-pcapng.xml | 185 +++++++++++++++++++++------------ 1 file changed, 121 insertions(+), 64 deletions(-) diff --git a/draft-tuexen-opsawg-pcapng.xml b/draft-tuexen-opsawg-pcapng.xml index f591abe..a33d344 100644 --- a/draft-tuexen-opsawg-pcapng.xml +++ b/draft-tuexen-opsawg-pcapng.xml @@ -1937,14 +1937,19 @@ Section Header
-
+
Alternative Packet Block offers an intermediate layout between - the Simple Packet Block and the Enhanced Packet Block. Just like the - Enhanced Packet Block it stores a timestamp of each packet, but - all other fields in the fixed part of the Alternative Packet - Block's body are made optional and can be stored in the Options - section. The packet's body of this block MAY be compressed. + the Simple Packet Block and the Enhanced Packet Block. The + timestamp of a packet, and other fields that are present in the + Enhanced Packet Block are made optional in the Alternative Packet + Block and can be stored in the Options section. The Options, + Timestmap, and packet's body of this block MAY be compressed. + This packet block type is intended for use cases that store large + number of packets, are concerned about storage size, and optionally + engage compression. This block type has minimal overhead of 16 + octets in storing packet data (size of: 'Block Type', + 2 * 'Block Total Length', 'APB Flags').
@@ -1952,75 +1957,151 @@ Section Header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +---------------------------------------------------------------+ - 0 | Block Type = 0x00000010 | + 0 | Block Type = 0x0000000A | +---------------------------------------------------------------+ 4 | Block Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - 8 | Timestamp (High) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -12 | Timestamp (Low) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -16 / / - / Options (variable) / - / / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - / / - / Packet Data / - / variable length, padded to 32 bits / - / / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 8 | APB Flags (apb_flags, code = 0x2001) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ \ +12 / / \ + / Options (variable) / + + / / | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | + | Optional Timestamp (High) | | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +- This part MAY + | Optional Timestamp (Low) | | be compressed + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | + / / | + / Packet Data / | + / variable length, padded to 32 bits / + + / / / + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ / | Block Total Length | +---------------------------------------------------------------+
- The permissible options listed in the table below are of - Simple Option type (see ) + The Alternative Packet Block has the following fields: + + Block Type: the block type of the Enhanced + Packet Block is 0x0A + + Block Total Length: total size of this block, + as described in + + APB Flags Option: Alternative + Packet Block Flags - MANDATORY option formatted as Simple Option + (see ) + with code 0x2001. The APB Flags indicate presence of a Timestamp, + uncompressed Options size, and characteristics of the Packet Data. + See for permissible bit settings. + This option MAY include total byte size of Options. If the size + is specified, then the opt_endofopt option is OPTIONAL. The value + of apb_opt_size is the number of octets comprising the entire + Options section in this block not counting the APB Flags. If + present, this option MUST be the first one in the Options data. + + Options: list of permissible option values listed in the table + below are of Simple Option type (see + ). The Options MAY be + compressed if the Compression Bits of the apb_flags are on. + + Optional Timestmap (High) and Timestamp (Low): upper 32 bits + and lower 32 bits of a 64-bit timestamp. If the Timestamp bit of + the apb_flags is on, then the Timestamp (High/Low) is present + after the Options. If there is no apb_flags option or the + Timestamp bit of the apb_flags option is off, then the timestamp + is not present. If the Compression bits of the apb_flags option + are on, then the Options, Timestamp, and Packet Data are + compressed using Compression Type specified in the apb_flags. + Otherwise it's not compressed. + The timestamp is a single 64-bit unsigned integer that + represents the number of units of time that have elapsed since + 1/1/1970 00:00:00 UTC. The length of a unit of time is specified + by the 'if_tsresol' option (see ) of + the Interface Description block referenced by this packet. + Note that, unlike timestamps in the libpcap file format, + timestamps in Enhanced Packet Blocks are not saved as two 32-bit + values that represent the seconds and microseconds that have + elapsed since 1/1/1970 00:00:00 UTC. Timestamps in Enhanced + Packet Blocks are saved as two 32-bit words that represent + the upper and lower 32 bits of a single 64-bit quantity. + + + + Bit Number + Description + + 0 + Timestamp bit (0 = no timestamp, 1 = has timestmap) + + 1-2 + Packet Direction. (00 = information not + available, 01 = inbound packet, 10 = outbound packet) + + 3-5 + Reception Type (000 = not specified, 001 = unicast, + 010 = multicast, 011 = broadcast, 100 = promiscuous). + + 6-11 + Compression Type. When specified, the optional Timestamp and + Packet Data contents are compressed. Permissible types: + 0 (uncompressed), 1 (lzw), 2 (gzip), 3 (bzip2), 4 (zip), + 5 (7z), 6 (lzo), 7 (ucl), 8 (snappy), other??? + + 12-15 + Reserved + + 16-23 + Total uncompressed byte size of Options in this section + excluding the APB Flags. This setting is OPTIONAL. If set to 0, + then opt_endofopt is REQUIRED. + + + Options are described in the table below: + Name Code Length Multiple allowed? - apb_opt_size - 0x2001 - - - no - apb_orig_len 0x2002 - no - apb_iface_id + apb_capt_len 0x2003 - no - apb_flags + apb_iface_id 0x2004 - no + - - Total byte size of options. If this option is specified, then - the opt_endofopt option is OPTIONAL. The value of apb_opt_size - is the number of octets comprising the entire Options section in - this block. If present, this option MUST be the first one in the - Options data. - - Packet's - Original Length. Actual length of the packet when it was + Original + Packet Length. Actual length of the packet when it was transmitted on the network. It can be different from captured - packet length, which is the min(snapshot length, bytes left in - the block), if the packet has been truncated by the capture + packet length if the packet has been truncated by the capture process. + Captured + Packet Length: number of octets captured from the packet (i.e. + the length of the uncompressed Packet Data field). It will be + the minimum value among the Original Packet Length and the + snapshot length for the interface (SnapLen, defined in + ). The value of this field does not + include the padding octets added at the end of the Packet Data + field to align the Packet Data field to a 32-bit boundary. + It specifies the interface this packet comes from; the correct interface will be the one whose Interface Description Block @@ -2030,33 +2111,9 @@ Section Header matching interface description block MUST exist. If this option is not defined, interface ID defaults to 0. - Alternative - Packet Block Flags indicate characteristics of the Packet Data. - See for permissible bit settings. - - Bit Number - Description - - 0-1 - Inbound / Outbound packet (00 = information not - available, 01 = inbound, 10 = outbound) - - 2-4 - Reception type (000 = not specified, 001 = unicast, - 010 = multicast, 011 = broadcast, 100 = promiscuous). - - 5-12 - Compression type. When specified, the Packet Data - content is compressed. Permissible types: - 0 (uncompressed), 1 (lzw), 2 (gzip), 3 (bzip2), 4 (zip), - 5 (7z), 6 (lzo), 7 (ucl), 8 (snappy), other??? - - 13-23 - Reserved -