Documentation request: Getting started on catalog builder
Documenting what I shared with a colleague. @Ciheim for the doc updates, please refer to these notes as needed to help a new user get started.
Project code
https://github.com/NOAA-GFDL/CatalogBuilder
Sphinx documentation
https://noaa-gfdl.github.io/CatalogBuilder/
Quick look at how the catalog generation calls look like:
1- https://github.com/NOAA-GFDL/CatalogBuilder/blob/main/catalogbuilder/scripts/gen_intake_gfdl_runner_config.py 2- This is a "testing" code used as part of the continuous integration tests. It creates sample data (empty)files. Here is where it's called in the CI pipeline. See example csv generated as one of the artifacts from the CI workflow under workflow-artifacts if the project allows access.
Youtube training
intake-esm concepts and such https://www.youtube.com/watch?v=H3AwVVBOfRE&t=1s The catalog builder tool relatedhttps://www.youtube.com/watch?v=84qmRI5BJJw&t=1s
Notebook examples
Preliminar notebooks examples to build catalog from a jupyter notebook, and to use them for analyzinghttps://github.com/aradhakrishnanGFDL/canopy-cats e.g. https://github.com/aradhakrishnanGFDL/canopy-cats/blob/main/notebooks/cm4_sample.ipynb
Notebooks from Pangeo cmip6 hackathon
https://gallery.pangeo.io/repos/pangeo-gallery/cmip6/the hackathon was held with cloud resources made available, which. no longer exists.
Meetings:
We have (hybrid) analysis meetings held bi-weekly on Wednesdays at 11AM. The meetings also discuss catalog integration into FRE.
Getting started:
While the docs show you multiple ways to get started, and the conda package is likely the cleanest..or the fre-cli if you're using it at GFDL, just describing how I run to quickly get things started in developer and reviewed mode. (fre-cli docs need to be improved, so users to keep an eye out on the sphinx doc updates)
clone the reposet up the conda environment using the environment.yml file
Follow the notebook example here once you set up your configuration file (example below) and your ipykernel.