Github Actions
Our open source project Host is on Github, and we are using its powerful Actions feature to do CICD, which you may not know by looking at Github Actions alone. In fact, it is what we often call CICD pipeline or workflow, when we Push code to Github, it will automatically trigger these pipelines. It will help us automatically build code, run test cases, build mirrors, publish mirrors, and so on. All of this is free.
Today I got the AgileConfig tests running on Github Actions. I've been having problems running integration tests on Actions, but I finally got it working today. Now that the test cases are working, can I see the code coverage directly in Actions? The answer is yes, of course.
Run tests and collect results
Each time we run a unit test, Microsoft's tools can actually generate the result description file for us. Please run the test using the following code:
dotnet test --no-build --verbosity normal --collect:"XPlat Code Coverage" --results-directory ./coverage
After the run is complete, we can see that several xml files will be generated, these files are actually used to describe the content and results of the test.
The test ran successfully.
Total number of tests: 123
Passed: 123
Total Time: 2.6757 minutes
1> Completed generating project "D:\00WORKSPACE\AgileConfig\" (VSTest target).
Successfully generated.
0 Warnings
0 errors
Used time 00:02:43.08
Attachment.
D:\00WORKSPACE\AgileConfig\coverage\1a7564b1-aa53-44ad-b34b-9abbb4b97c9f\
D:\00WORKSPACE\AgileConfig\coverage\45f571fb-2ec4-445b-b444-6879f784c925\
D:\00WORKSPACE\AgileConfig\coverage\ff1d49a5-9663-42f9-813b-2ef81fe5ed3f\
D:\00WORKSPACE\AgileConfig\coverage\6b502f8f-a99f-4931-8643-faf79db7273c\
D:\00WORKSPACE\AgileConfig\coverage\ff389298-7b68-4e60-a35c-d9c0db8cd640\
Analyze test results locally
With these xml files we can analyze them. Of course, with the naked eye to analyze can not see what. Let's analyze it locally with the plugin.
Install the plugin in VS:
Fine Code Coverage
After installing the plugin, run through all the test cases again. In the output window of VS we selectFCC 。
This window prints out the test results for each project.
By opening "View" > "Other windows" > "Fine code coverage" we can view the results in more detail.
Using CodeCoverageSummary
Since we can analyze these files locally, it's obvious that we just need to be able to insert a step-by-step analysis into our Github Actions. Obviously, some of the big guys have already implemented this Action, so we just need to integrate it into our own workflow file.
CodeCoverageSummary：
A GitHub Action that reads Cobertura format code coverage files from your test suite and outputs a text or markdown summary. This summary can be posted as a Pull Request comment or included in Release Notes by other actions to give you an immediate insight into the health of your code without using a third-party site.
CodeCoverageSummary
Modify our CI Workflow file:
- name: Build
run: dotnet build --no-restore
- name: Test
run: dotnet test --no-build --verbosity normal --collect:"XPlat Code Coverage" --results-directory ./coverage
- name: Code Coverage Report
uses: irongut/[email protected]
with:
filename: coverage/**/
badge: true
fail_below_min: true
format: markdown
hide_branch_rate: false
hide_complexity: true
indicators: true
output: both
thresholds: '60 80'
It is very easy to use. And there are very many configuration items, not one by one here. If you need to look directly at its documentation.
Let's run this workflow for a while.
Coverage File: /github/workspace/coverage/d497ebaf-5670-41ea-8215-dfb4f3a037d6/
Coverage File: /github/workspace/coverage/83632b7c-8df2-4179-b0a8-e6410a71b1b7/
Coverage File: /github/workspace/coverage/b434d3bb-7909-4a7d-b0f2-f8620d429cfc/
Coverage File: /github/workspace/coverage/af81da37-b5c5-4edd-a409-74984355857e/
Coverage File: /github/workspace/coverage/1b159b39-3eb4-4951-936e-afde68925b34/
Package | Line Rate | Branch Rate | Health
-------- | --------- | ----------- | ------
| 41% | 67% | ❌
| 0% | 100% | ❌
| 95% | 93% | ✔
| 72% | 59% | ➖
| 28% | 21% | ❌
| 0% | 0% | ❌
| 73% | 54% | ➖
| 77% | 50% | ➖
| 0% | 100% | ❌
| 67% | 88% | ➖
| 97% | 100% | ✔
| 1% | 0% | ❌
| 95% | 67% | ✔
| 45% | 31% | ❌
| 80% | 75% | ✔
| 89% | 100% | ✔
| 1% | 0% | ❌
| 77% | 67% | ➖
| 0% | 100% | ❌
| 1% | 0% | ❌
| 21% | 25% | ❌
| 27% | 17% | ❌
| 0% | 100% | ❌
| 0% | 0% | ❌
| 0% | 0% | ❌
| 2% | 2% | ❌
| 0% | 0% | ❌
| 0% | 0% | ❌
| 0% | 0% | ❌
| 3% | 100% | ❌
| 0% | 0% | ❌
| 15% | 100% | ❌
| 0% | 0% | ❌
| 2% | 3% | ❌
| 0% | 0% | ❌
| 4% | 2% | ❌
**Summary** | **18%** (2009 / 15061) | **15%** (337 / 3196) | ❌
_Minimum allowed line rate is `60%`_
FAIL: Overall line rate below minimum threshold of 60%.
We found that this workflow run failed. The reason is that our coverage is only 18%. We have just configured the minimum value to be 60%, and if it is lower than 60, the workflow will be interrupted with an exception.Looking at the logs, we can also see the statistics for each project. There are line rate, branch rate, health and so on.
summarize
This time we demonstrated how to analyze the results of unit tests locally using the VS plugin to get coverage, and how to analyze the results of tests within the Github Action workflow using CodeCoverageSummary. It's still very simple. I hope it will be helpful to you.
Well enough of that, off to catch up on unit tests.
Follow my public number to play with technology together
