(!) Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

Training an ISM using the Combinatorial-Trainer

Description: This tutorial explains how the combinatorial trainer can be used to train an ISM from a recorded scene.

Tutorial Level: BEGINNER

Next Tutorial: How to use the recognizer

Setup

ou need a recorded scene stored in a sqlite-database for this tutorial; see recorder tutorial on how to create one.

Tutorial

Basic training

Before you can launch the trainer you will have to set the parameters of the application. At least this will be the path to the sqlite-database containing the recorded scene and the path to the output directory which will contains the trainer's results.

The files containing the parameters of the combinatorial trainer can be found in the launch-directory of the asr_ism package. Open the combinatorialTrainer.launch file and set the value of the outputDataPath parameter to the path of a directory you want to store your results in, e.g. "/home/username/comb_trainer_results/". Make sure this directory exists or the trainer will fail to execute.

Next go to the param-directory and open the sqlitedb.yaml file. Here you will need to set the path to the database your recorded scene is in; if you went through the SceneRecorder tutorial and want to use the database you created, you can use the same value as you did there. Otherwise set dbfilename to the correct path to your database (also make sure the parameter is uncommented or the trainer will fail).

Now you can start the application by using the following command:

roslaunch asr_ism combinatorialTrainer.launch

If you set all parameters correctly the trainer should give you a console output similar to the following:

comb_trainer_output.png

Wait until the trainer is done and the application exits, then you can use the trained model which can be found in the SQL-subdirectory of the output-directory you set before. The name of it should be optimized.sqlite.

Advanced topics

The section before described how a model is trained using the default parameters. For advanced users the application offers a list of parameters to influence the training further. The documentation for all of them can be found in the combinatorialTrainer.launch file (they can be set there as well) or here, this section gives a short overview of the most important ones:

As the search for an optimal topology can be seen as a combinatorial problem, the algorithms this trainer uses fall into this category. More specifically this application offers three algorithms the user can make use of to solve this problem by adjusting the optimizationAlgorithmId parameter:

  1. Hill climbing (value: 0)
    • Parameters for this algorithm: hillClimbingStrictMonoton, randomRestartProbability, randomWalkProbability

  2. Simulated annealing (value: 1)
    • Parameters for this algorithm: startTemperature, endTemperature, temperatureFactor, repetitionsBeforeUpdated

  3. Record hunt (value: 2, default)

    • Parameters for this algorithm: initialAcceptableCostDelta, costDeltaDecreaseFactor

In each optimization round (starting with a generated or manually chosen start topology) the trainer generates neighbor topologies of the current best one by either adding a relation between objects, removing one (can be turned off with removeRelations) or swapping one relation (can be turned off with swapRelations), then it chooses the optimal one depending on the optimization algorithm and a cost function which is a weighted sum of the average runtime of this topology (evaluated with a generated or provided test set) and the false positives of it. The weights for this sum can be tuned by setting the parameters alpha and beta.

For further evaluation the start-topology and fully-meshed one can be stored in addition to the resulting optimized one by setting the parameters storeStartTopologyISM and storeFullyMeshedISM.

Output files

The application generates several output files which are grouped in subdirectories in your output directory:

  1. CSV: Contains two files with information about the training process:

    1. A file containing meta data (from left to right: number of evaluated topologies, number of optimization rounds, elapsed runtime)
    2. A file containing information about the chosen topologies (each line representing a single one) during the optimization process (from left to right: index of the optimization round, number of false positives, elapsed runtime)
  2. Logfile: Contains a text-file which includes the log of the application. This is equivalent to the console output during the runtime of the application.

  3. SQL: The resulting database containing the trained, optimized ISM can be found here (optimized.sqlite). Depending on your parameters the ISM of the fully meshed topology and/or the one of the start topology can be found here as well.

  4. SVG: A visualization of the training process (more specifically the optimization of the used topology) can be found here.

    • The image is read from bottom to the top, each circle representing one generated topology during the optimization process. The ones in the bottom section are the generated starting topologies and the one with the blue border is the chosen best one of them. Each following section contains the generated neighbor topologies again with the best one visualized with a blue border. All topologies chosen during the training are connected with a blue line. The color of a circle represents the costs of this topology with green being cheap and red being expensive; the size represents the average runtime (smaller means smaller runtime). Inside the circles there are information about the specific topology (from top to bottom: the id of the topology, its exact cost value, the number of false positives and the average evaluation runtime)
    • Example (because no neighbor-topology was better than the one with id 2 the optimization stops there):

    • vis_scene.png

  5. Topologies: Contains information about all valid topologies used during the training process. Each one is saved in a seperate folder and has the following files describing it:

    1. A .csv file which contains the number of false positives and false negatives which occured during evaluation, as well as the average evaluation time
    2. A text file containing the name of the scene and the relations of the topology (this file can be used as an input parameter for the trainer describing the start topology to use in further training executions)
    3. Two .dot files which can be used for visualization purposes

Wiki: asr_ism/tutorial_combinatorial_trainer (last edited 2019-11-02 18:14:44 by TobiasAllgeyer)