Topic on Talk:Continuous integration/Codehealth Pipeline

Unit vs Integration test coverage

2
Summary by KHarlan (WMF)

Resolved per

GLederrey (WMF) (talkcontribs)

There is a whole discussion to be had about unit vs integration tests and what coverage should be measured / reported. I would argue that unit tests should be measured in ratio of branch covered and integration tests should be measured in ratio of functional requirements covered (a much harder metric to automate). Mixing those 2 metrics is somewhat problematic and distract from the goal of unit testing, which should be improving modularity more than improving correctness.

Jdforrester (WMF) (talkcontribs)

I agree (both that the metric for unit coverage isn't really the right one for functionality test coverage, and that it's hard to measure "how much functionality is tested").

However, I'd suggest that there's a worthwhile distinction between integration tests (unit X works with unit Y1 and also unit Y2) and system tests (end-to-end functionality collectively performs as expected in multiple configurations), so it's even more messy.