Skip to content

Design: precedence contract for a consumer-supplied PesterConfiguration (#83) #156

Description

@tablackburn

Part of #120 (Phase 3 — API improvements). Blocks #83.

Question

When a consumer supplies their own [PesterConfiguration] object via $PSBPreference.Test.Configuration, what wins — their object, or the existing $PSBPreference.Test.* settings?

#83 raises this and leaves it open. It is the one genuinely undesigned item left in v1.0.0, and it defines public API surface that freezes at 1.0.0 — getting the precedence wrong is a breaking change to undo later.

Note: #120 previously pointed at "joshooaj's design in #80". That pointer is stale — #80 is a different, already-closed request about Output.Verbosity and Run.SkipRemainingOnFailure. There is no existing design to implement.

Open questions

  1. Precedence. Does the supplied object act as a base that $PSBPreference.Test.* overrides, or as a final say that PowerShellBuild only fills gaps in? A consumer who sets both and gets the opposite of what they expected is the failure mode to design against.
  2. Run.Path. PowerShellBuild computes the test path today. If the consumer sets Run.Path on their object, is it honored, ignored, or an error? The advanced surface permits multiple paths, which the current single-path model has no answer for.
  3. Opinionated defaults. Which settings does PowerShellBuild insist on regardless (code coverage output location, result file paths, CI-detection behavior) because the surrounding build tasks depend on them?
  4. Discoverability. If a supplied setting is silently overridden, how does the consumer find out? Silent override is the worst outcome — worse than an error.
  5. Pester 5 and 6. Test-PSBuildPester supports both majors and the test: Add Test-PSBuildPester integration tests and fix latent bugs #137 matrix verifies both. Does the configuration object surface differ between them in ways that affect the answer?

How to resolve

Conversation, not code — /grilling plus /domain-modeling. @joshooaj should be in it: they raised #83 and the design questions are theirs. A prototype of the merge logic is reasonable if the discussion stalls on something concrete.

Done when

The precedence contract is written down here in enough detail that #83 can be implemented without further design decisions.

Activity

added this to the v1.0.0 milestone on Aug 20, 2026

tablackburn commented on Aug 26, 2026

@tablackburn
ContributorAuthor

Evidence brief (2026-08-26)

Assembled so the design conversation starts from facts. This decides nothing — every option
below is left open with its trade-off, and questions 1–5 still need @joshooaj and @tablackburn.

The finding that changes the shape of the question

Pester already ships the merge primitive, and it is identical in both supported majors.

[PesterConfiguration]::Merge($base, $override)   # public static; verified on 5.7.1 and 6.1.0

The override wins only for options whose IsModified is $true; untouched options fall
through to the base; neither input is mutated. Pester uses it on itself in src/Main.ps1:

[PesterConfiguration] $PesterPreference = [PesterConfiguration]::Merge([PesterConfiguration]::Default, $Configuration)

So no merge logic needs writing. The decision is which object is base, which is override, and
what gets re-asserted afterward.

The constraint that rules out the obvious answer

Test-PSBuildPester.ps1:95-100 assigns six properties unconditionally, and Pester marks an
option IsModified on any assignment — even assigning its existing default. Meanwhile
build.properties.ps1:47-109 gives every $PSBPreference.Test.* key a concrete value; there is no
unset state.

Therefore "$PSBPreference.Test.* overrides the supplied object" would override a consumer's
Output.Verbosity with 'Detailed' even when they set nothing else — precisely the failure
mode question 1 says to design against. That option only works alongside giving those preferences a
$null/unset state, which changes the documented default table in README.md:105-119.

Question 3 — the premise needs correcting

The issue asks which settings the surrounding build tasks depend on. Reading psakeFile.ps1 and
IB.tasks.ps1 end to end: they depend on none. Neither reads a test-result or coverage file
afterward. Task Test is an empty aggregator.

Every hard dependency is internal to Test-PSBuildPester, and there are four:

Property Why Severity
Run.PassThru = $true line 111 assigns $testResult, line 116 gates on $testResult.Result. There is no Set-StrictMode anywhere in the module, so a consumer setting $false makes $null.Result -eq 'Failed' false — a build with failing tests passes silently not negotiable
CodeCoverage.OutputPath line 122-125 re-reads the file and parses it as JaCoCo high
CodeCoverage.OutputFormat same parser high
CodeCoverage.Enabled gates both blocks medium

TestResult.* is not structurally required — nothing reads testResults.xml. It can be
consumer-overridable at zero cost.

There is also no CI detection anywhere in the module, so question 3's "CI-detection behavior"
has no current implementation to preserve.

Worth noting for scoping: the coverage gate could read $testResult.CodeCoverage instead of
re-parsing the XML — this repository's own root psakeFile.ps1:89-95 already does exactly
that
— which would shrink the frozen 1.0.0 contract from four properties to one. Bigger change,
much smaller public contract.

Question 2 — Run.Path is not set at all

Test-PSBuildPester never assigns Run.Path. It relies on Push-Location -LiteralPath $Path
(line 92, popped at 155) plus Pester's default Run.Path = '.'.

Two consequences: a consumer's relative Run.Path would resolve against Test.RootDir rather than
the project root, and honoring Run.Path properly means removing the Push-Location pair — which
also moves where the coverage file at line 122 resolves. It cannot be bolted onto a merge-only
implementation.

The single-path limit is ours, not Pester's: Run.Path is already StringArrayOption in both
majors. Widening [string]$Path to [string[]] is not a breaking change for callers.

Question 4 — nothing today

Test-PSBuildPester.ps1 contains zero Write-Warning and zero Write-Verbose. No prior art was
found for warning on override — Pester, Maester, and PSModule/Invoke-Pester all override silently.

IsModified makes a precise warning cheap if scoped to the reserved set: you can enumerate
exactly what the consumer actually set. Scoped to everything PowerShellBuild assigns, it would fire
on all six unconditional properties every build.

Question 5 — measured, and the answer is encouraging

New-PesterConfiguration compared in separate processes, Pester 5.7.1 vs 6.1.0:

  • 44 properties vs 56; nothing removed in 6
  • all nine properties PowerShellBuild sets exist in both, same types, same defaults
  • exactly one shared default differs: CodeCoverage.UseBreakpoints (5: True → 6: False)
  • twelve Pester-6-only properties, including Run.Parallel, Run.Shuffle, Should.DisableV5

A version-agnostic contract is feasible, provided it is expressed generically — ::Merge plus a
named reserved list — rather than as an enumerated property table.

Prior art

Maester is the closest analog, and its answer to question 2 is worth reading
(Invoke-Maester.ps1:286-304):
consumer object is the base; the tool overrides unconditionally only the two it structurally
needs — including Run.PassThru, the same property we hardcode — and conditionally, only when its
own parameter was supplied
, for path and filters. Silently.

Pester itself merges then corrects individual settings afterward (src/functions/Output.ps1
special-cases -not $PesterPreference.Output.Verbosity.IsModified to raise verbosity under CI).

Two things found on the way, both separate from this design

  • IB.tasks.ps1:100 reads $PSBPreference.Test.CodeCoverage.OutputFormat, but
    build.properties.ps1:102 defines OutputFileFormat.
    Invoke-Build users splat $null.
    psakeFile.ps1:161 has it right. Filed separately.
  • The declared Pester floor of 5.0.0 is already optimistic. New-PesterConfiguration landed in
    5.2 and Run.SkipRemainingOnFailure — assigned unconditionally at line 97 — in 5.3. Worth raising
    to 5.3.0 as part of whatever ships here.

Also: tests/Test-PSBuildPester.tests.ps1 contains no references to PesterConfiguration, so
nothing existing pins the current assignments. A merge change has no test to break — which cuts both
ways.

tablackburn commented on Aug 28, 2026

@tablackburn
ContributorAuthor

Deferring to 1.1.0 along with #83, which this exists to unblock.

The design questions here are real and none of them are answered — that is exactly the problem. They need @joshooaj, who raised #83, and the conversation has not started. Meanwhile the finished 1.0.0 break surface (PlatyPS 1.x, psake 5.0.4, Pester 6.0.0, the PowerShell 5.1 floor) has been sitting unreleased since 0.8.2 waiting behind it.

Nothing about this design gets harder by shipping 1.0.0 first. The surface precedence has to interact with, $PSBPreference.Test.*, has been frozen since 0.8.2 either way — the version number on the next release does not constrain it.

Everything already written here stands; the five open questions are still the right five. This is a scheduling change, not a change of scope or intent. See #83 for the fuller reasoning.

modified the milestones: v1.0.0, v1.1.0 on Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestwayfinder:grillingWayfinder ticket: resolved by conversation with a human

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions