Overview

Version 2.0 of the ROS bag file format adds a number of new features:

  • compression: messages are stored in chunks which can be individually compressed, while still allowing random-access

  • higher-level index: a second level of indexing allows for tools to quickly gather statistics on the bag file

  • connection-oriented: messages are stored by connection, making republishing of recorded messages more faithful

  • compact: the format has been trimmed down, e.g. message timestamps are stored at almost half their previous size; message records don't contain duplicated topic and connection information

Format

A bag file consists of sequences of records, with an initial line to indicate the format version number:

#ROSBAG V2.0
<record 1><record 2>....<record N>

Records

NOTE: the record format is unchanged from version 1.2.

Each record has the following format:

<header_len><header><data_len><data>

Name

Description

Format

Length

header_len

length, in bytes, of the header to follow

little-endian integer

4 bytes

header

metadata for the record

(see below)

header_len bytes

data_len

length, in bytes, of the data to follow

little-endian integer

4 bytes

data

record specific data

(see below)

data_len bytes

Headers

Each record header (the header field that precedes the data field) contains a sequence of name=value fields, formatted as follows:

<field1_len><field1_name>=<field1_value><field2_len><field2_name>=<field2_value>...<fieldN_len><fieldN_name>=<fieldN_value>

Name

Description

Format

Length

fieldX_len

length, in bytes, of the name and value to follow

little-endian integer

4 bytes

fieldX_name

name of the field

character string

variable; terminated by '='

fieldX_value

value of the field

field-specific

variable

The total length of fieldX_name=fieldX_value, including the '=' character, is fieldX_len bytes. The total length of this header sequence (including the names, '=' characters, value lengths, and values) is header_len bytes.

Header fields may appear in any order. Field names can contain any printable ASCII character (0x20 - 0x7e), except '=' (0x3d). Field values can contain any data (including binary data with embedded nulls, newlines, etc.)

Op codes

Every header is required to include the following field:

Name

Description

Format

Length

op

record type identifier (see below)

unsigned byte

1 byte

The op field is used to distinguish between different types of record. The following op values are valid (listed in order of their first appearance in a bag):

  • Bag header. Stores information about the entire bag, such as the offset to the first index data record, and the number of chunks and connections.

  • Chunk. Stores (possibly compressed) connection and message records.

  • Connection. Stores the header of a ROS connection, including topic name and full text of the message definition.

  • Message data. Stores the serialized message data (which can be zero-length) with the ID of the connection.

  • Index data. Stores an index of messages in a single connection of the preceding chunk.

  • Chunk info. Stores information about messages in a chunk.

Records

Bag header

The bag header record occurs once in the file as the first record.

The following fields are guaranteed to appear in the bag header record (op=0x03):

Name

Description

Format

Length

index_pos

offset of first record after the chunk section

little-endian long integer

8 bytes

conn_count

number of unique connections in the file

little-endian integer

4 bytes

chunk_count

number of chunk records in the file

little-endian integer

4 bytes

The bag header record is padded out by filling data with ASCII space characters (0x20) so that additional information can be added after the bag file is recorded. Currently, this padding is such that the header is 4096 bytes long.

Chunk

The following fields are guaranteed to appear in a chunk record (op=0x05):

Name

Description

Format

Length

compression

compression type for the data

character string

variable

size

size in bytes of the uncompressed chunk

little-endian integer

4 bytes

The supported compression values are "none" and "bz2". The compressed size of the chunk can be found in the data_len field of the record.

The data for a chunk record consists of message data and connection records, compressed using the method specified in the chunk record header.

Connection

The following fields must appear in a connection record header (op=0x07):

Name

Description

Format

Length

conn

unique connection ID

little-endian integer

4 bytes

topic

topic on which the messages are stored

character string

variable

The data consists of a string containing the connection header in the same format as a bag record header. The following fields must appear in the connection header: topic, type, md5sum, message_definition. Optional fields include: callerid, latching.

Two topic fields exist (in the record and connection headers). This is because messages can be written to the bag file on a topic different from where they were originally published.

Message data

The following fields are guaranteed to appear in a message data header (op=0x02):

Name

Description

Format

Length

conn

ID for connection on which message arrived

little-endian integer

4 bytes

time

time at which the message was received

little-endian long integer

8 bytes

The data in these records is the serialized message data in the ROS serialization format.

Index data

The following fields are guaranteed to appear in an index data header (op=0x04):

Name

Description

Format

Length

ver

index data record version

little-endian integer

4 bytes

conn

connection ID

little-endian integer

4 bytes

count

number of messages on conn in the preceding chunk

little-endian integer

4 bytes

The data in these records depends on the version in the header. The current version is version 1, which consists of count repeating occurrences of timestamps, chunk record offsets and message offsets:

Name

Description

Format

Length

time

time at which the message was received

little-endian long integer

8 bytes

offset

offset of message data record in uncompressed chunk data

little-endian integer

4 bytes

Chunk info

The following fields are guaranteed to appear in a chunk info header (op=0x06):

Name

Description

Format

Length

ver

chunk info record version

little-endian integer

4 bytes

chunk_pos

offset of the chunk record

little-endian long integer

8 bytes

start_time

timestamp of earliest message in the chunk

little-endian long integer

8 bytes

end_time

timestamp of latest message in the chunk

little-endian long integer

8 bytes

count

number of connections in the chunk

little-endian integer

4 bytes

The data in these records depends on the version in the header. The current version is version 1, which consists of count repeating occurrences of connection ID's and message counts:

Name

Description

Format

Length

conn

connection id

little-endian long integer

4 bytes

count

number of messages that arrived on this connection in the chunk

little-endian long integer

4 bytes

Wiki: Bags/Format/2.0 (last edited 2013-09-06 23:35:55 by TullyFoote)