When the result is not a measurement: weld seams with an ordinal and a nominal response

Step 2 / Planning

How many plates does a judgement need?

A category carries less information than a measurement. A single plate only says “porosity” or “no porosity” — the probability behind it only emerges from many. A full factorial with three levels per factor has 27 settings; the question is how often each has to be welded.

The table shows how often the analysis is usable when the whole study is drawn a hundred times from the true models. “Usable” means: the model converges, every category occurs, and no estimate runs off to infinity.

The ordinal model is frugal — it estimates one slope per factor, the same for all steps. The nominal one is not: it estimates separate slopes for each of the three defects, twelve parameters instead of six. With one plate per setting it is unusable in three studies out of four, with two still in one out of five. From three plates per setting it is 96%, and a fourth adds little. The choice is therefore 81 plates.

What “running off to infinity” means is separation: if a rare defect never occurs for a group of settings, its probability there can be pushed arbitrarily close to zero, and the estimate grows without bound. DoEStat reports this in the analysis (“separation suspected”). The problem grows with every additional term — a squared gas term, which suggests itself in a three-level design, leaves only 45 of 100 nominal analyses usable at 81 plates in the same simulation, instead of 96. The model here therefore has main effects only.

This planning calculation is done by the example itself, by repeating the study from the true models. The power analysis in DoEStat covers numeric and binomial responses, not categorical ones with more than two levels.

Step 4 / Spatter

Ranked steps: the proportional-odds model

For a ranked response DoEStat chooses the cumulative logit model: it describes the probability of reaching at most a given step. One can picture an invisible spatter tendency behind the four steps that the factors shift; three cut-points divide it into the steps. Each factor gets one slope that applies to all three cut-points — hence “proportional odds”.

A positive slope pushes the seam towards more spatter. The current acts strongly (z = 5.7), the shielding gas dampens (p = 0.008), the travel speed is not detectable (p = 0.33). The column “True value” is the comparison with the model the data come from — the current is somewhat overestimated, the direction and ranking are right.

Does the assumption of equal slopes hold?

The model claims that a factor acts equally strongly at every cut-point. Whether the data support this is checked by the Brant test: it estimates the three cut-points separately and compares the slopes. A small p-value would mean that a factor acts differently between “none” and “few”, say, than between “moderate” and “many” — the multinomial model would then be the more honest choice.

No factor objects (jointly p = 0.52). The ordinal model may stay — and for three factors it needs three slopes instead of nine.

The confusion matrix compares the observed step with the most likely step according to the model. 48% hits sound low, but that is the nature of a probability: at a setting with 45% “few” and 35% “none”, “few” is the best prediction and still often wrong. “Moderate” is never the most likely step — it lies between two neighbours that are each more frequent. What the model separates reliably are the ends: no plate with “no” spatter is predicted as “moderate” or “many”, and vice versa.

Step 6 / Process window

Where a good seam is most likely

The probability profiler shows, for each category, how its probability changes when one factor is moved and the others are held. The four curves in a column add up to one everywhere. In DoEStat the lines are dragged with the mouse and both responses are seen at once.

The profilers stand at the setting at which a good plate — no weld defect, at most a few spatters — is most likely. It was found as the product of the two probabilities over a grid of the region:

The current lies in the lower quarter of the range: more current brings spatter and undercuts, less brings lack of fusion. Travel speed and shielding gas sit on the edge of the region — welding slowly with plenty of gas is simply better here. Anyone searching beyond that leaves the region studied; the model knows nothing there.

Summing up

What the right scale achieved

Two responses that cannot be measured, only classified, and for each the model that fits its kind. The ordinal model uses the order of the spatter steps and makes do with three slopes; the Brant test confirmed that this is enough. The multinomial model gives each weld defect its own cause — and that was exactly what was needed, because the current prevents one defect and causes another.

The price is the number of plates. Categorical data need replicates, and the nominal model needs more than the ordinal one. The planning calculation beforehand showed that 54 plates would have led to no usable result in one study out of five, and that an obvious squared term makes matters worse.

Against the truth: all factors that act in the true model are found, with the right sign; all that do not act stay inconspicuous. The estimates deviate from the truth by up to a little over half a logit — with 81 plates and one judgement each, that is the precision one can expect. And the confirmation shows that the probability at the best setting is somewhat overestimated: anyone who has to make a commitment takes the result of the confirmation plates, not the prediction.