<> == Overview == `rosdoc_rosorg` is an extension of the [[rosdoc]] tool to support generating resources necessary for the ROS.org web site. This includes various indexes used by wiki macros and other tools. This package is not intended for general use, though people interested in setting up their own indexing infrastructure are welcome to contact the author for help setting this up on their site. == Version-controlled files == Rosdoc_rosorg as well as ros_release depend on several files stored in version control of those projects, some of which are backups of generation results, others are maintained manually. * https://code.ros.org/svn/release/trunk * distros/*.rosdistro: * '''manually maintained''' * is read by: * rospkg.distro.distro_uri() * ros_release/job_generation... (several references) * ros_release/rosdistro/src/rosdistro.py * distros/*.excludes * '''manually maintained''' * is read by: * ros_release/release/scripts/is_excluded * ros_release/rosdeb/scripts/list_missing * release_resources/rosinstalls * '''manually maintained''' * ''no reference found, seems cturtle was last time it was used'' * release_resources/repos * '''manually maintained''' * ''no reference found, seems cturtle was last time it was used'' * rosdoc_rosorg * rosdoc_rosorg/repos.list * list of repo groups (all, electric, unstable?) * '''manually maintained''' * rosdoc_rosorg/repos.fuerte.list * list of all known repo groups for fuerte * '''manually maintained''' * rosdoc_rosorg/repos/*.rosinstall * referenced by repos.[*]list * define sets of SCM repositories * '''manually maintained''' * rosdoc_rosorg/index/repos-*.rosinstall * '''manually maintained''' * rosinstall file as to create a workspace against apt-get installed distros * rosdoc_rosorg/index/distro-*.yaml * '''redundant''', only restructures data * '''generated''' solely from release/distros/*.rosdistro using create_distro_index.py * used for rosdoc_rosorg --distro-index === Generated files not under version control === * repos.rosinstall * '''generated''' from repos.list and therein referenced repos/*.rosinstall * is a single huge rosinstall file as the sum of all that went into it * created using generate_index.py * used by rosdoc_rosorg --repos * http://www.ros.org/browse/megamanifest.xml * http://www.ros.org/browse/megamanifest.yaml * http://www.ros.org/browse/megastack.yaml generated by rosdoc_rosorg --rosbrowse, all used by rosbrowse, from distro*.yaml (I guess always the latest & greatest?) * http://www.ros.org/browse/list.php {{attachment:rosorg.png}} Graphical overview of scripts and data files. Simple rectangles represent data files. When given with path, those are under version control. Other shapes are (python) scripts. scripts consume files and produce files. --options show how to pass an input to a script, or what option is required to produce output. == scripts == === create_distro_index.py === Usage: create_distro_index.py Writes into a yaml dict: {{{ {'repos': safe_repo_index, 'stacks': stack_index, 'distro_name': distro_name} }}} based on rospkg.distro.load_distro(), which in turn reads files from code.ros.org/svn/release..../.rosdistro The transformation does not add any information, it just applies rules to generate uris. So this generated all the .yaml files that were placed into rosorg_rosdoc/index. === generate_index.py === usage: generate_index.py [options] writes into rosinstall (default repos.rosinstall), based on rospkg.distro and https://code.ros.org/svn/ros/stacks/rosorg/trunk/rosdoc_rosorg/repos.list Source file contains lists of pairs (uri: foo, name: bar.rosinstall) where each bar.rosinstall lists repositories from which to check out sources. === rosdoc_rosorg === Usage: rosdoc_rosorg --repos= [options] [packages...] {{{ Options: -h, --help show this help message and exit -n NAME, --name=NAME Name for documentation set -q, --quiet Suppress some errors --paths=PATHS package paths to document -o OUTPUT_DIRECTORY directory to write documentation to --upload=RSYNC_TARGET rsync target argument --repos=ROSBROWSE_REPOS_FILE rosinstall file with repos list for determining repository names/roots --distro-index=DISTRO_INDEX_FILE distro index for determining repository contents for binary installs --checkout=CHECKOUT_DIR path to checkout directory for repos file --test run in test mode --rosbrowse run rosbrowse indexer instead }}} --repos is a required option, it takes a rosinstall-like file with additional "meta" tags to declare membership of an SCM repository to a larger "repo" (e.g. all lab-wide stacks). --distro-index is an optional file (e.g. rosdoc_rosorg/distro-*.yaml) : {{{ distro_name: foo repos: repo-name: [stackname1, stackname2] stacks: stackname1: {version: ver1, repo: repo1, rules: {git: &id001 {uri: foo, dev-branch: bar, distro-tag: dist1, release-tag: rtag1} } } stackname2: {version: ver1, repo: repo1, rules: {git: *id001 } } }}} Which contains redundant structural data (repo <-> stack). Also it contains yaml references (*0001 points to &0001) The code is parsed by core.py, where the Repo class represents one stack (not Repo), and a repo is represented as an aggregate name of a stack. --checkout allows to pass a location where stacks have been checked out locally, instead of being apt-get installed released stacks. --rosbrowse is a switch. If it is off, the script will use rosdoc to generate API docs for all stacks and packages discovered using roslib.rospack, and also create for each repo listed in the file given in --repos option (and distro-index option), manifest.yaml, stack.yaml and repo.yaml for each repo discovered by parsing the given files. If --rosbrowse is given, instead of the repo.yaml, stack.yaml and manifest.yaml, the megamanifest and megastack are generated as megamanifest.xml, megamanifest.yaml and megastack.yaml. Those can be uploaded to: * http://www.ros.org/browse/megamanifest.xml * http://www.ros.org/browse/megamanifest.yaml * http://www.ros.org/browse/megastack.yaml The --rosbrowse option generating mega* files ignores the list of packages given as arguments, those only affect doc generation. Basically the --rosbrowse switch should be a separate command / script. Other rosdoc_rosorg options relate to the original rosdoc script. === install_distro_index === usage: install_distro_index.py calls sudo apt-get install with each stackname mentioned in the YAML file dict 'stacks'. ## AUTOGENERATED DON'T DELETE ## CategoryPackage