####################################
##FILL ME IN
####################################
## links to any required tutorials
## note.0= [[ROS/Tutorials|ROS tutorials]]
## descriptive title for the tutorial
## title = Analyzing Diagnostic Logs Off-Line
## multi-line description to be displayed in search 
## description = This tutorial uses the [[diagnostic_analysis]] package to analyze a robot log file. These files are generated automatically by the robots when they're running and contain the entire diagnostics history of the robot. You can convert these log files into CSV files (comma separated value) that can be opened with any spreadsheet editor. This only works with <<MsgLink(diagnostic_msgs/DiagnosticArray)>> messages on the `/diagnostics` topic, used in the PR2 diagnostic toolchain.
## the next tutorial description 
## next =
## links to next tutorial
## next.0.link=
## next.1.link=
## what level user is this tutorial for
## level= BeginnerCategory
####################################

<<IncludeCSTemplate(TutorialCSHeaderTemplate)>>

<<TableOfContents(4)>>

=== Types of Log Files ===

If you have any messages that aren't on the "/diagnostics" topic, they will be discarded (NOTE: The leading slash in the topic name is important; make sure to bag "/diagnostics", not "diagnostics").
{{{
Discarding message on topic: /turtle1/pose
Discarding message on topic: /turtle1/color_sensor
Discarding message on topic: /turtle1/pose
Discarding message on topic: /turtle1/color_sensor
}}}
The PR2 diagnostics toolchain uses <<MsgLink(diagnostic_msgs/DiagnosticArray)>> messages on the "/diagnostics" topic, which is what this package is designed for. Only [[Bags|bag files]] of this type will work with the [[diagnostic_analysis]] package. To play other bag files, use [[rostopic]].

=== Recover Log File ===

To recover the data from a robot's log file, look in the /hwlog folder on your pr2. Log files end with ".bag", and are recorded directly with the diagnostics data from the robot.

{{{
$ cd /hwlog
}}}
Files are stored with the label 'prX_runtime_automatic_YYYY-MM-DD-HH-MM-SS-topic.bag'.

You may want to copy these to a directory off the robot.  E.g., if your computer is ''xxx'' and your username is "user":

{{{
$ scp prX_runtime_automatic_YYYY-MM-DD-HH-MM-SS-topic.bag user@xxx:<somewhere_you_want_it_path>
}}}

If the log file you're looking for isn't there, check in the folder:
{{{
stor6:/prdata/bag_repository
}}}
Log files are moved there during nightly syncs. 

=== Convert Log Files to CSV's ===

Robot log files can be viewed with ordinary spreadsheet software after processing with the CSV exporter.  When run on a bag file, this creates several CSV files into the `output/log_filename` directory.

{{{
$ rosrun diagnostic_analysis export_csv.py <bagfilename>
}}}

You can use the "-d" option to change the output directory. For example:
{{{
rosrun diagnostic_analysis export_csv.py <bagfilename> -d <dir>
}}}
will store the results in the "<dir>/output" directory.

Each "name" in the diagnostic status messages will have its own file. This means you'll see one file for the Power Board, another for EtherCAT Master, etc. 

=== Thinning the Logs ===

Some bag files are too large to be viewed with ordinary spreadsheet software. The limit for Open Office and versions of Excel prior to Excel 2007 is 65,536 rows. If you have a long run to analyze, it may help to make the CSV "sparse", by only retaining every nth line.

To do this, use sparse_csv.py. 
{{{
$ rosrun diagnostic_analysis sparse_csv.py <path/my_bagfile>.csv -m
}}}
This makes the file <path/my_bagfile>_sparse.csv, making it about 65000 lines max. 

=== Analyzing ===

`LibreOffice Calc` is the preferred tool.  For example,

{{{
$ localc output/BAG_DIRECTORY/<my_bagfile>.csv
}}}

opens the data in `LibreOffice Calc`. From there, you can analyze and graph data.

## TutorialCategory
## DiagnosticsCategory
## DiagnosticAnalysisCategory