5.1 NETLOG File Format

The NETLOG file format is used by SAM in both its logging and analysis functionality, and also by FCM when it logs queue size information. The format must be:

• Easily written, as it must be written in near real-time.
• Easily converted to other file formats using awk.
• Easily inspected by hand.

The last two requirements restrict the NETLOG file format to a text based file format, since binary files are neither easily inspected by hand, nor easily manipulated using awk, although they would be faster to write and smaller in size. A text based file format also makes it difficult for files to be corrupted, and allows manual inspection to help in debugging and correctness.

A NETLOG file consists of a version identifier, a header line, followed by one or more records. Each of these is separated by a newline character, which will appear only following the version identifier, a header or between records.

The version identifier for the file format specified here will be NETLOG1.0

The header line has the following format:

HEAD:(' ', FIELDNAME)+

That is, it begins with the string HEAD: and is followed by one or more field names with a space before each. A field name is simply a string identifying the field. This file format places no limitations on possible fields. For a list of fields used by FCM and SAM, see sections 5.1.3 and 5.1.4.

Each record consists of $n$ data items, each separated by a space, where $n$ is the number of field names specified in the header of the file.

5.1.1 Example NETLOG file

NETLOG1.0
HEAD: stream size window
0 521 32768
1 700 16000
0 521 29643
0 521 29643
0 521 25223
1 700 32768

5.1.2 NETLOG Grammar

file -> version, newline, header, (newline, record)+
version -> NETLOG1.0
header -> 'HEAD:', (' ', fieldname)+
record -> field, (' ', field)*
fieldname -> alphan+ | '(', ')'
field -> digit+
digit -> '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9'
newline -> '\n'

5.1.3 FCM Generated NETLOG files

The logging component of FCM is used to generate NETLOG files based on information about the queue. The following information will be recorded, with the field identifiers given.

1. Time - The time in milliseconds. This will be a relative time only, and not an absolute time corresponding to a particular time and date. All times shall be non-negative integers, and 0 will be the time during the logger's initialisation. The identifier for this field is time.
2. Queue Size (Bytes) - The current size of the Router's queue, recorded in Bytes. This is the number of Bytes within the packets stored in the queue at the particular moment in time. The identifier for this field is qbytes.
3. Queue Size (Packets) - The number of packets in the Router's queue. This is the number of packets which are waiting in the queue to be sent at the Router. The identifier for this field is qpackets.
4. $p(q)$ - The value of $p(q)$. This is the value of $p(q)$ which would be sent in a packet were one to be sent at this particular moment. As the logging of queue information is performed at regular time intervals, this does not have to match up with a value of $p(q)$ which is actually transmitted to a Receiver. The identifier for this field is p(q).
5. Number Of Packets Transferred - The number of packets transferred from the Router across the low bandwidth link since the last record. As the Router only supplies the number of packets transferred since it began, this will be the difference since the last reading of the Router's value. Note that this value includes retries and other unsuccessful transmissions. The identifier for this field is packets.

5.1.4 SAM Generated NETLOG files

When the logger component of SAM is used to generate NETLOG files, the user has the option of selecting any or all of the following fields to be recorded.

1. Time - The time in milliseconds. This is the same field as specified for Time in section 5.1.3. Again the identifier is time. This time corresponds to the time the packet was received at the network interface, as recorded by libpcap.
2. IP Packet Length - The length of the IP packet. This is the length of the IP packet, as specified in the length field of the IP header. The identifier for this field is iplength.
3. Protocol - The value of the protocol field in the IP header. This determines whether the packet is a TCP, UDP, ICMP, etc. packet. The identifier for this field is ipprotocol.
4. Stream - The stream in the corresponding STREAMSLOG (see section 5.2) file to which this packet belongs. This will be 0 for packets which do not belong to such a stream. The identifier for this field is stream.
5. Sequence Number - The sequence number of a TCP packet. This is the value recorded in the sequence number field of the TCP header. For non-TCP packets this will be 0. The identifier for this field is tcpsequence.
6. Acknowledgement Number - The acknowledgement number of a TCP packet. This is the value recorded in the acknowledgement number field of the TCP header. For non-TCP packets this will be 0. The identifier for this field is tcpacknowledge.
7. Window - The window size as announced in the TCP packet header. This may or may not be a value calculated using the Client's algorithm. The identifier for this field is tcpwindow.
8. Number of Packets - The number of packets from which the data for the record was derived. When the interval is 1 packet, this will always be 1. The identifier for this field is packets.
9. Throughput - The throughput of the TCP stream. This is based on the amount of data, and length of time of the last 64 packets. The identifier for this field is throughput.
10. Round Trip Time - The time taken for data to travel from one computer back to the other. This is calculated by calculating the time taken for acknowledgements to arrive. The identifier for this field is rtt.