Skip to content

Add stable decomposition clarification #957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fdwr merged 2 commits into from
Oct 10, 2026
Merged

Add stable decomposition clarification #957

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dc5a1bd
Add stable decomposition clarification
mklimenko-nv Sep 14, 2026
42a3dc4
Add clarification comment
mklimenko-nv Oct 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 22 additions & 11 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Status Text: <p>
<li>it has at least two independent, interoperable implementations of every feature defined in the specification, where interoperability can be verified by passing open test suites, and two or more implementations interoperating with each other;</li>
<li>it has an open test suite of every feature defined in the specification.</li>
</ul>
Text Macro: EMULATED generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
Text Macro: EMULATED generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. This decomposition is an illustrative example rather than a reference implementation: it does not prescribe the algorithm or the precision of intermediate results. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
</pre>
<pre class="anchors">
urlPrefix: https://tc39.es/ecma262/; spec: ECMA-262
Expand Down Expand Up @@ -9066,7 +9066,15 @@ partial dictionary MLOpSupportLimits {
}

function reduceLogSumExp(builder, input, options) {
return builder.log(builder.reduceSum(builder.exp(input), options));
// This is numerically more stable than the direct decomposition
// log(reduceSum(exp(input))): subtracting the maximum keeps the
// exponentials in range, and the maximum is added back at the end.
const maxX =
builder.reduceMax(input, {axes: options.axes, keepDimensions: true});
const sumExpX =
builder.reduceSum(builder.exp(builder.sub(input, maxX)), options);
return builder.add(
builder.log(sumExpX), builder.reshape(maxX, sumExpX.shape));

@fdwr fdwr Sep 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like adding some comment explaining how a "reduce log sum" became a "reduceMax" would be enlightening. e.g.

// This is more numerically stable than a direct decomposition:
// return builder.log(builder.reduceSum(builder.exp(input), options));

@mklimenko-nv mklimenko-nv Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, added a comment in 42a3dc4

}

function reduceSumSquare(builder, input, options) {
Expand Down Expand Up @@ -10350,8 +10358,12 @@ partial dictionary MLOpSupportLimits {
</summary>
<pre highlight="js">
function softplus(builder, input) {
return builder.log(
builder.add(builder.exp(input), builder.constant(input.dataType, 1)));
const zero = builder.constant(input.dataType, 0);
const one = builder.constant(input.dataType, 1);
return builder.add(
builder.max(zero, input),
builder.log(
builder.add(one, builder.exp(builder.neg(builder.abs(input))))));
}
</pre>
</details>
Expand Down Expand Up @@ -10661,13 +10673,12 @@ partial dictionary MLOpSupportLimits {
</summary>
<pre highlight="js">
function tanh(builder, input) {
return builder.div(
builder.sub(
builder.exp(builder.mul(builder.constant(input.dataType, 2), input)),
builder.constant(input.dataType, 1)),
builder.add(
builder.exp(builder.mul(builder.constant(input.dataType, 2), input)),
builder.constant(input.dataType, 1)));
const one = builder.constant(input.dataType, 1);
const expNeg2AbsX = builder.exp(
builder.mul(builder.constant(input.dataType, -2), builder.abs(input)));
return builder.mul(
builder.sign(input),
builder.div(builder.sub(one, expNeg2AbsX), builder.add(one, expNeg2AbsX)));
}
</pre>
</details>
Expand Down
Loading