Contents
Overview
Version 2.0 of the ROS bag file format adds a number of new features:
compression: messages are stored in chunks which can be individually compressed, while still allowing random-access
higher-level index: a second level of indexing allows for tools to quickly gather statistics on the bag file
connection-oriented: messages are stored by connection, making republishing of recorded messages more faithful
compact: the format has been trimmed down, e.g. message timestamps are stored at almost half their previous size; message records don't contain duplicated topic and connection information
Format
A bag file consists of sequences of records, with an initial line to indicate the format version number:
#ROSBAG V2.0 <record 1><record 2>....<record N>
Records
NOTE: the record format is unchanged from version 1.2.
Each record has the following format:
<header_len><header><data_len><data>
Name |
Description |
Format |
Length |
header_len |
length, in bytes, of the header to follow |
little-endian integer |
4 bytes |
header |
metadata for the record |
(see below) |
header_len bytes |
data_len |
length, in bytes, of the data to follow |
little-endian integer |
4 bytes |
data |
record specific data |
(see below) |
data_len bytes |
Headers
Each record header (the header field that precedes the data field) contains a sequence of name=value fields, formatted as follows:
<field1_len><field1_name>=<field1_value><field2_len><field2_name>=<field2_value>...<fieldN_len><fieldN_name>=<fieldN_value>
Name |
Description |
Format |
Length |
fieldX_len |
length, in bytes, of the name and value to follow |
little-endian integer |
4 bytes |
fieldX_name |
name of the field |
character string |
variable; terminated by '=' |
fieldX_value |
value of the field |
field-specific |
variable |
The total length of fieldX_name=fieldX_value, including the '=' character, is fieldX_len bytes. The total length of this header sequence (including the names, '=' characters, value lengths, and values) is header_len bytes.
Header fields may appear in any order. Field names can contain any printable ASCII character (0x20 - 0x7e), except '=' (0x3d). Field values can contain any data (including binary data with embedded nulls, newlines, etc.)
Op codes
Every header is required to include the following field:
Name |
Description |
Format |
Length |
op |
record type identifier (see below) |
unsigned byte |
1 byte |
The op field is used to distinguish between different types of record. The following op values are valid (listed in order of their first appearance in a bag):
Bag header. Stores information about the entire bag, such as the offset to the first index data record, and the number of chunks and connections.
Chunk. Stores (possibly compressed) connection and message records.
Connection. Stores the header of a ROS connection, including topic name and full text of the message definition.
Message data. Stores the serialized message data (which can be zero-length) with the ID of the connection.
Index data. Stores an index of messages in a single connection of the preceding chunk.
Chunk info. Stores information about messages in a chunk.
Records
Bag header
The bag header record occurs once in the file as the first record.
The following fields are guaranteed to appear in the bag header record (op=0x03):
Name |
Description |
Format |
Length |
index_pos |
offset of first record after the chunk section |
little-endian long integer |
8 bytes |
conn_count |
number of unique connections in the file |
little-endian integer |
4 bytes |
chunk_count |
number of chunk records in the file |
little-endian integer |
4 bytes |
The bag header record is padded out by filling data with ASCII space characters (0x20) so that additional information can be added after the bag file is recorded. Currently, this padding is such that the header is 4096 bytes long.
Chunk
The following fields are guaranteed to appear in a chunk record (op=0x05):
Name |
Description |
Format |
Length |
compression |
compression type for the data |
character string |
variable |
size |
size in bytes of the uncompressed chunk |
little-endian integer |
4 bytes |
The supported compression values are "none" and "bz2". The compressed size of the chunk can be found in the data_len field of the record.
The data for a chunk record consists of message data and connection records, compressed using the method specified in the chunk record header.
Connection
The following fields must appear in a connection record header (op=0x07):
Name |
Description |
Format |
Length |
conn |
unique connection ID |
little-endian integer |
4 bytes |
topic |
topic on which the messages are stored |
character string |
variable |
The data consists of a string containing the connection header in the same format as a bag record header. The following fields must appear in the connection header: topic, type, md5sum, message_definition. Optional fields include: callerid, latching.
Two topic fields exist (in the record and connection headers). This is because messages can be written to the bag file on a topic different from where they were originally published.
Message data
The following fields are guaranteed to appear in a message data header (op=0x02):
Name |
Description |
Format |
Length |
conn |
ID for connection on which message arrived |
little-endian integer |
4 bytes |
time |
time at which the message was received |
little-endian long integer |
8 bytes |
The data in these records is the serialized message data in the ROS serialization format.
Index data
The following fields are guaranteed to appear in an index data header (op=0x04):
Name |
Description |
Format |
Length |
ver |
index data record version |
little-endian integer |
4 bytes |
conn |
connection ID |
little-endian integer |
4 bytes |
count |
number of messages on conn in the preceding chunk |
little-endian integer |
4 bytes |
The data in these records depends on the version in the header. The current version is version 1, which consists of count repeating occurrences of timestamps, chunk record offsets and message offsets:
Name |
Description |
Format |
Length |
time |
time at which the message was received |
little-endian long integer |
8 bytes |
offset |
offset of message data record in uncompressed chunk data |
little-endian integer |
4 bytes |
Chunk info
The following fields are guaranteed to appear in a chunk info header (op=0x06):
Name |
Description |
Format |
Length |
ver |
chunk info record version |
little-endian integer |
4 bytes |
chunk_pos |
offset of the chunk record |
little-endian long integer |
8 bytes |
start_time |
timestamp of earliest message in the chunk |
little-endian long integer |
8 bytes |
end_time |
timestamp of latest message in the chunk |
little-endian long integer |
8 bytes |
count |
number of connections in the chunk |
little-endian integer |
4 bytes |
The data in these records depends on the version in the header. The current version is version 1, which consists of count repeating occurrences of connection ID's and message counts:
Name |
Description |
Format |
Length |
conn |
connection id |
little-endian long integer |
4 bytes |
count |
number of messages that arrived on this connection in the chunk |
little-endian long integer |
4 bytes |