A skip is not a pass¶

go test without -v prints the same line whether a package ran its tests or skipped every one of them:

ok  gitlab.com/phpboyscout/phpbotscout/pkg/embed  1.810s   <- 38 tests skipped
ok  gitlab.com/phpboyscout/phpbotscout/pkg/embed  12.263s  <- 38 tests ran

Both are green. The first is a pipeline reporting success for work that never happened, and nothing on the screen says so. On one day in August that shape turned up five times in unrelated repositories: embedding tests skipping for want of a runtime, a web build exiting 0 and shipping a placeholder UI, a republish run that published nothing, configuration that did nothing against the image it targeted, and two of this repository's own self-tests passing while the engine they tested had never started. Each was cheap to catch and nobody had written the catch, because an absence does not show up in a log.

phpbotscout's pkg/embed is the measured case. When its integration tests started running instead of skipping, the package's coverage went from 52.0% to 91.2% and its test time from 1.8s to 12.3s. The pipeline was green on both sides of that change.

Two reasons not to run, and only one is legitimate¶

A suite that needs something heavy (a native library, a container, a network service) is gated behind an environment variable, INT_TEST or a narrower INT_TEST_<GROUP>, so that a developer's go test ./... stays fast. That gives the test two ways not to run:

The second case is a contradiction, and skipping hides it. A CI job that sets the gate and then silently runs nothing looks exactly like a job whose tests all passed.

The check belongs in the test helper¶

The distinction needs intent, and only the test knows the intent. So the guard goes in the helper every gated test already calls, right after the gate check. From phpbotscout !53, with its comments and the full failure message trimmed:

if os.Getenv("INT_TEST") != "1" && os.Getenv("INT_TEST_EMBED") != "1" {
    t.Skip("INT_TEST_EMBED is not 1")
}

lib := os.Getenv(libEnv)
if lib == "" {
    t.Fatalf("%s is not set, but the integration gate is on", libEnv)
}

Six lines, no CI plumbing, and it behaves the same on a laptop as in a pipeline. Any repository that gates a suite this way has the exposure until its helper does the same.

Why not count skips in CI instead¶

A job that fails when tests skip looks like the general fix, and it is the wrong one. It sees only that a test skipped, never why. The ordinary case is the gate being off, which is most runs and every developer's machine, so the counter fires constantly, and a check that fires constantly is disabled within a week. After that it is worse than no check, because its presence suggests coverage it no longer gives.

What a component can do safely is show the numbers: which tests skipped and why, a passed / skipped / failed count, and a JUnit report GitLab turns into a Tests tab. That fails nothing, so it cannot misfire, and it turns "you had to know to look" into something on the screen. It is proposed for go-test in spec 0101, and is not a substitute for the guard above.