good first issueperf
Repository metrics
- Stars
- (392 stars)
- PR merge metrics
- (PR metrics pending)
Description
Fp max reductions would typically look like:
for(nvfuser_index_t i154 = 0; i154 < 8; ++i154) {
int i299;
i299 = 4 * i154;
#pragma unroll
for(nvfuser_index_t i156 = 0; i156 < 4; ++i156) {
T29[0] = fmax(
T29[0],
T24[(i299 + i156)]);
}
}
Here, fmax is is not just fmaxf, but it also incurs two more comparisons in case the arguments are NAN: https://github.com/NVIDIA/Fuser/blob/main/runtime/helpers.cu#LL102C1-L111C2
This could be translated as:
bool is_nan = false;
#pragma unroll
for(nvfuser_index_t i154 = 0; i154 < 8; ++i154) {
int i299;
i299 = 4 * i154;
#pragma unroll
for(nvfuser_index_t i156 = 0; i156 < 4; ++i156) {
#if 0
T29[0] = fmax(
T29[0],
T24[(i299 + i156)]);
#else
T29[0] = T29[0] > T24[(i299 + i156)] ? T29[0] : T24[(i299 + i156)];
is_nan = is_nan || isnan(T24[(i299 + i156)]);
#endif
}
}
if (is_nan) {
T29[0] = NAN;
}
In the case of cross entropy loss (#278), I observed 20% speedup on A100.
I think this translation could be done automatically as part of lowering. See the translation for welford vectorization.