Code coverage of testing suite
We recently started evaluating the code coverage of our testing suite. We're following a setup `@angus-g` made using gfortran and lcov, and then posting the results to codecov.io for analysis.
[](https://codecov.io/gh/NOAA-GFDL/MOM6)
The testing represented by the ocean_only suite of 47 examples covers only ~50% lines of code under MOM6/src. Adding 6 ice_ocean_SIS2 experiments has no discernible effect on MOM6 coverage.
Tasks:
1. Tweak existing experiments so that every option is used at least once, e.g. apparently [we have no experiments using implicit fifth order edge slopes with PQM](https://codecov.io/gh/NOAA-GFDL/MOM6/src/a3778b73c1fb69f9aac7191a3c7f56ad3c421fec/src/ALE/regrid_edge_slopes.F90#L182).
2. Set up a _gitlab_ pipeline to periodically re-analyze the HEAD of dev/master (since it takes a while, we can't have this occur for every commit, nor run it on free resources such as Travis).
Considerations:
- Running one test experiment does not exercise all the code due to the plethora of run-time
options. Hence we need to run multiple tests with different configurations of run-time parameters.
- Code coverage depends most directly on the suite of configurations than on anything else.
- The way gfortran instruments the code results in the executable writing files to the build directory.
- This means the experiments/tests can only be run sequentially, one after the other.
- For accuracy of code analysis we should not use optimization, making the running slow.
- ocean_only require 72 cores to complete (takes ~45 min on c4 running sequentially).
- ice_ocean_SIS2 includes OM4_025 so requires 480 cores (takes an additional 20 minutes) but did not appear to alter coverage.
- DEBUG=True, DEBUG_BT=True took ~4 hrs for just ocean_only and accounts for ~1.5% of code.
- FATAL/WARNING code will never be reached unless we actively mis-configure the model. I am not sure how to measure what fraction of the code this represents.
issue