[mohawk2]

Description

Currently, to override BLAS/LAPACK's handling of errors involves defining a xerbla_ (or sometimes xerbla) symbol in your library/executable. This may not work in 2-level namespace environments like macOS.

Instead, this PR allows you to call SET_XERBLA (Fortran) with a replacement handler, or NULL() to reset to default. There is also an GET_(library)_XERBLA. This does not address that the new handler must be a Fortran-compatible routine.

There are CBLAS (cblas_set_xerbla) and LAPACKE (LAPACKE_set_xerbla) handlers on the way, but I'm putting this up for review as I believe it already adds value.

I have marked below that the documentation has been updated, because I added a paragraph in the doc-comments of xerbla.f. If more is needed, please let me know.

EDIT: updated function names.

Checklist

• The documentation has been updated.
• If the PR solves a specific issue, it is set to be closed on merge.

[codecov]

Codecov Report

❌ Patch coverage is 85.60000% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.72%. Comparing base (c9713d3) to head (43d0daa).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
CBLAS/src/cblas_xerbla.c	50.00%	5 Missing and 2 partials ⚠️
LAPACKE/utils/lapacke_xerbla.c	80.95%	2 Missing and 2 partials ⚠️
BLAS/SRC/xerbla_blas.f90	83.33%	2 Missing ⚠️
CBLAS/src/xerbla.c	77.77%	1 Missing and 1 partial ⚠️
SRC/xerbla_lapack.f90	83.33%	2 Missing ⚠️
CBLAS/testing/c_xerbla.c	50.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1407   +/-   ##
=======================================
  Coverage   69.72%   69.72%           
=======================================
  Files        6204     6206    +2     
  Lines      497194   497229   +35     
  Branches    23430    23415   -15     
=======================================
+ Hits       346652   346686   +34     
  Misses     150116   150116           
- Partials      426      427    +1     

Components	Coverage Δ
BLAS	99.76% <86.66%> (+<0.01%)	⬆️
CBLAS	97.07% <60.86%> (+0.08%)	⬆️
LAPACK	82.66% <86.66%> (+<0.01%)	⬆️
LAPACKE	2.17% <80.95%> (+0.01%)	⬆️
TMGLIB	55.69% <ø> (ø)
BLAS testing	88.77% <100.00%> (+<0.01%)	⬆️
CBLAS testing	89.62% <92.30%> (-0.02%)	⬇️
LAPACK testing	82.25% <100.00%> (+<0.01%)	⬆️
LAPACKE testing	∅ <ø> (∅)

Files with missing lines	Coverage Δ
BLAS/SRC/xerbla.f	71.42% <100.00%> (+71.42%)	⬆️
BLAS/TESTING/cblat2.f	85.47% <100.00%> (+0.01%)	⬆️
BLAS/TESTING/cblat3.f	90.97% <100.00%> (+<0.01%)	⬆️
BLAS/TESTING/dblat2.f	85.38% <100.00%> (+<0.01%)	⬆️
BLAS/TESTING/dblat3.f	89.45% <100.00%> (+<0.01%)	⬆️
BLAS/TESTING/sblat2.f	85.38% <100.00%> (+<0.01%)	⬆️
BLAS/TESTING/sblat3.f	89.45% <100.00%> (+<0.01%)	⬆️
BLAS/TESTING/zblat2.f	85.46% <100.00%> (+<0.01%)	⬆️
BLAS/TESTING/zblat3.f	90.97% <100.00%> (+<0.01%)	⬆️
CBLAS/testing/c_c2chke.c	93.35% <100.00%> (+0.03%)	⬆️
... and 38 more

... and 1 file with indirect coverage changes

---

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9713d3...43d0daa. Read the comment docs.

[ACSimon33]

Hi @mohawk2,

I like the idea a lot, since it would solve all the problems we currently have with error testing (e.g., it doesn't work with shared Windows libs or two-level namespaces) and make it easier for users to use a custom xerbla on all platforms.

A few things to consider:

• Naming: There are precedents from other BLAS/LAPACK libraries (openblas_set_xerbla, mkl_set_xerbla). Also, it might make sense to split the function into two versions, one for BLAS and one for LAPACK, like this: BLAS_SET_XERBLA and LAPACK_SET_XERBLA (or with REGISTER instead of SET). That would cause fewer name symbol clashes, and the user could decide which error handler to replace.
• Return pointers: openblas_set_xerbla and mkl_set_xerbla return the previous xerbla function pointer. I'm not sure yet how easy that would be to replicate when implementing it in Fortran instead of C...
• Testing optimized libs: Linking the testing framework with optimized BLAS/LAPACK would not work until vendors implement it. Not really an issue, since if they want to use the reference test suite, they should implement the missing functions.
• CBLAS/LAPACKE: It would be nice to have corresponding CBLAS and LAPACKE versions that set their xerbla functions. I don't think anything in the PR makes that difficult, so this can be a follow-up discussion and PR.

[ilayn]

There is also a possibility of defining it as a weak symbol so tests can override the regular one. that can limit the surgery to only xerbla. overall I don't think xerbla is a good way to report errors back but it is what it is for now.

[ACSimon33]

There is also a possibility of defining it as a weak symbol so tests can override the regular one. that can limit the surgery to only xerbla. overall I don't think xerbla is a good way to report errors back but it is what it is for now.

Yes, that's how it's currently done in the CBLAS test suite. Won't work on Windows, though. Also, doing that in Fortran would depend on the compilers supporting it (nagfor doesn't without major hacks, ifx & flang maybe).

[ilayn]

I do use it on windows albeit on a C translation and Clang.

Also, doing that in Fortran would depend on the compilers supporting it (nagfor doesn't without major hacks, ifx & flang maybe).

ah then nevermind. Sorry for the noise

[ACSimon33]

I do use it on windows albeit on a C translation and Clang.

Ah, you're right; Clang and Intel do support it on Windows for static libraries. Did you try it with a shared library? Because last time I checked, that was the blocker.

[mohawk2]

Thank you all for your quick responses! As noted in my edit of the PR description, I'm intending to add CBLAS and LAPACKE equivalents. I'll ponder the names, I hadn't properly checked prior art and will do so now.

I'll ponder the idea of different BLAS and LAPACK handlers - is it really true that the current implementation essentially has two similarly-named symbols that are each getting overridden by the user-supplied one? If so, that seems horrible. I might need to make differently-named entry points for each that call each other, since the user expectation will be a full override for both.

[ACSimon33]

I'll ponder the idea of different BLAS and LAPACK handlers - is it really true that the current implementation essentially has two similarly-named symbols that are each are getting overridden by the user-supplied one? If so, that seems horrible. I might need to make differently-named entry points for each that call each other, since the user expectation will be a full override for both.

Yes, I think using the same REGISTER_XERBLA symbol name in BLAS and LAPACK could mean it only replaces one of the handlers (if we use separate shared libraries). My sugestion would be to have the separate symbols and then one symbol that does both and lives in LAPACK. Something like this:

    BLAS_SET_XERBLA      ->  libblas.so
    LAPACK_SET_XERBLA    ->  liblapack.so
    SET_XERBLA           ->  liblapack.so (calls the other two)
    
    BLAS_RESET_XERBLA    ->  libblas.so
    LAPACK_RESET_XERBLA  ->  liblapack.so
    RESET_XERBLA         ->  liblapack.so (calls the other two)

Edit:

One issue would be that when we are compiling LAPACK with a vendor BLAS lib and they don't implement BLAS_SET_XERBLA yet, SET_XERBLA would call an undefined symbol. So we would need to check if the supplied vendor BLAS contains the symbol and, if not, disable the call via a preprocessor define.

[mohawk2]

Yes, I think using the same REGISTER_XERBLA symbol name in BLAS and LAPACK could mean it only replaces one of the handlers (if we use separate shared libraries). My sugestion would be to have the separate symbols and then one symbol that does both and lives in LAPACK. Something like this:

    BLAS_SET_XERBLA      ->  libblas.so
    LAPACK_SET_XERBLA    ->  liblapack.so
    SET_XERBLA           ->  liblapack.so (calls the other two)
    
    BLAS_RESET_XERBLA    ->  libblas.so
    LAPACK_RESET_XERBLA  ->  liblapack.so
    RESET_XERBLA         ->  liblapack.so (calls the other two)

My pondering came up with a similar result, but not identical (see below). The above discussion included the point that prior art (openblas_set_xerbla, where Google tried real hard to persuade me I really meant openblas set_xlabel) returns the previously-set handler. That's fine in C, where you can just ignore a return value, but (I looked it up, but felt I should confirm it) in Fortran you cannot ignore return values, which would render code using the SET function quite ugly. Also, having two separate values would render a function (which must return one value) unworkable. I have come up with this:

Library	Name	Type	Purpose
BLAS	SET_BLAS_XERBLA	Subroutine	Sets BLAS one; use NULL() value to get default
BLAS	GET_BLAS_XERBLA	Subroutine	Returns current BLAS one
LAPACK	SET_XERBLA	Subroutine	Calls SET_LAPACK_XERBLA and, if available, SET_BLAS_XERBLA
LAPACK	SET_LAPACK_XERBLA	Subroutine	Sets LAPACK one; use NULL() value to get default
LAPACK	GET_LAPACK_XERBLA	Subroutine	Returns current LAPACK one

Edit 1: specify that NULL() gives default behaviour in SET_*.
Edit 2: rename so SET_/GET_ is at start, to avoid cognitive load in CBLAS/LAPACKE versions.
Edit 3: GET_ must be a subroutine because no mixing ENTRY types (ref, plus I tried it).
Edit 4: Prototype code that works (see https://blogs.perl.org/users/ed_j/2026/09/adventures-with-fortran.html).
Edit 5: Moved prototype code to a blog post.
Edit 6: Removed BLAS SET_XERBLA as would create dependency on LAPACK.
Edit 7: Having switched to modules, we can now have GET_BLAS_XERBLA et al be functions.
Edit 8: Update SET_XERBLA to clarify SET_BLAS_XERBLA only called if available.
Edit 9: GET_BLAS_XERBLA et al can't be functions, thanks to an ifx bug (see discussion below).

One issue would be that when we are compiling LAPACK with a vendor BLAS lib and they don't implement BLAS_SET_XERBLA yet, SET_XERBLA would call an undefined symbol. So we would need to check if the supplied vendor BLAS contains the symbol and, if not, disable the call via a preprocessor define.

Yes, that sounds like a good approach. (Edit: now implemented, see below)

[mohawk2]

My thinking so far on a CBLAS approach (The LAPACKE version will be mutatis mutandis, so will not be addressed separately). CBLAS has an actual xerbla_ replacement with C-orientated enhancements, which I'd need to update to have the same as each new respective xerbla.f part, so that if Fortran code linked with our Brave New BLAS called set_xerbla etc, its expectations would be met.

cblas_set_xerbla reveals a close-to-ambiguity, so I've updated the above to move the library name portion of the procedures to after the verb.

For the CBLAS versions of the 3 procedures plus cblas_xerbla, the handler interface will be the same as cblas_xerbla (including varargs stuff). Other notes:

Function	Return value	Behaviour
LAPACKE_set_xerbla	Previous value of LAPACKE handler, like openblas_set_xerbla	Calls LAPACKE_set_lapack_xerbla and, if available, cblas_set_blas_xerbla
cblas_get_blas_xerbla	Current CBLAS handler	Return current
cblas_set_blas_xerbla	Current CBLAS handler	Will have a xerbla_ wrapper to pass to SET_BLAS_XERBLA_, that will refer internally to a callback. It will need to call SET_BLAS_XERBLA_ explicitly with a callback since we won't now rely on the symbol override, and it might not have worked. Details will become clearer on implementation.
cblas_xerbla	void	Will be updated like xerbla_ to dispatch on whether the stored callback pointer is NULL.

[mohawk2]

Yes, I think using the same REGISTER_XERBLA symbol name in BLAS and LAPACK could mean it only replaces one of the handlers (if we use separate shared libraries). My suggestion would be to have the separate symbols and then one symbol that does both and lives in LAPACK. Something like this:
[...]
One issue would be that when we are compiling LAPACK with a vendor BLAS lib and they don't implement BLAS_SET_XERBLA yet, SET_XERBLA would call an undefined symbol. So we would need to check if the supplied vendor BLAS contains the symbol and, if not, disable the call via a preprocessor define.

@ACSimon33 I've had to adjust the plan a bit; a BLAS SET_XERBLA would have a dependency on LAPACK, which I cannot see as a good thing, so I've removed that. However there is now in the latest push an LAPACK SET_XERBLA which seems to work fine.

[ACSimon33]

@mohawk2 Can you please merge the current master back into your branch? We now fixed numerical issues and errors on all platforms that are tested in CI, and the CI jobs are now configured to fail if new errors occur.

[mohawk2]

@mohawk2 Can you please merge the current master back into your branch? We now fixed numerical issues and errors on all platforms that are tested in CI, and the CI jobs are now configured to fail if new errors occur.

Working on this now, thank you for highlighting this.

[mohawk2]

@ACSimon33 @langou I've just rebased this to latest master and force-pushed; it passes all the tests on my local device. It doesn't have the probe for a compliant BLAS yet, nor CBLAS/LAPACKE implementations, which I'll be working on now, but I believe it is ready for merging as-is.

[mohawk2]

@ACSimon33 @langou Sorry about this! I'm pretty sure the Makefile jobs in CI will fail, because I missed updating the Makefiles to add the 2 new files due to inept git grep. Will add another commit in a few moments.

[mohawk2]

@ACSimon33 @langou Pushed. Works locally (and confirmed that without the new commit, it broke).

[mohawk2]

Flang is failing with:

error: loc("/home/runner/work/lapack/lapack/BLAS/SRC/xerbla_blas.f90":74:47): 'hlfir.declare' op operand #0 must be any reference or box, but got '!fir.boxproc<() -> ()>'
error: loc("/home/runner/work/lapack/lapack/BLAS/SRC/xerbla_blas.f90":74:47): 'hlfir.declare' op operand #0 must be any reference or box, but got '!fir.boxproc<() -> ()>'
error: loc("/home/runner/work/lapack/lapack/BLAS/SRC/xerbla_blas.f90":74:47): 'hlfir.declare' op operand #0 must be any reference or box, but got '!fir.boxproc<() -> ()>'

Some preliminary searching indicates this is because it doesn't like the combination of procedure pointers, and ENTRY. I've stuck with ENTRY because the _64 generator doesn't handle modules right. I can make it do so; it's extra work but I don't mind.

@ACSimon33 How do you want to proceed? If you can't accept breaking flang, which I get, please say so and I'll make a small sub-PR of this work so far with the 2 Generate* improvements and the noise-reduction commit that changes the test xerblas to calling their own xer_replace.

Edit: also codecov is saying it's failing because this part doesn't get exercised:

   if (.not. associated(active_callback)) then
     print *, 'Error: LAPACK_XERBLA called but no callback registered'
     stop
   end if

I had that in there because I thought it was better than a SEGV. I'm not sure how to proceed there.

[mohawk2]

@ACSimon33 (sorry for all the highlights) I have opened #1411 further to my last comment, and will get started on instead using module and enhancing the Generate to handle that.

[ACSimon33]

@mohawk2 Try removing the GET_ entries. They seem to be the problem. It builds fine for me without them.

[mohawk2]

@ACSimon33 I may need some help here; I'm getting these results at the end for some, which I don't know how to deal with:

• openmp-windows-2025-gfortran (shared)
• windows-2025-vs2026-ifx (shared)
• windows-2025-vs2026-flang (shared)
• windows-2025-gfortran (shared)

Quick analysis: it's only on Windows, and only with shared. The errors are both with _64 and without.

Edit: false alarm; it's only that those are having the usual two-level problem, and BLAS's xerbla isn't getting overridden by CBLAS's implementation for BLAS-internal errors. I'll investigate a way, which may involve exposing a hook to override the BLAS xerbla, or making the BLAS xerbla sit in its own little shared library, which can then be overridden by the CBLAS one.

[ACSimon33]

@mohawk2 Do we maybe want to finish the Fortran side, i.e. BLAS and LAPACK, first and tackle the CBLAS, LAPACKE side in a follow-up PR? Smaller PRs are better to review and defend. If that's not possible we can also put it in one, but I'd prefer two.

[mohawk2]

@mohawk2 Do we maybe want to finish the Fortran side, i.e. BLAS and LAPACK, first and tackle the CBLAS, LAPACKE side in a follow-up PR? Smaller PRs are better to review and defend. If that's not possible we can also put it in one, but I'd prefer two.

@ACSimon33 I was keen for that a little while ago, but my feeling now is a) I'm pretty close to finishing CBLAS (I think I know how to fix the Windows side), and the LAPACKE side looks clear to me; b) the CBLAS/LAPACKE part have influenced the design and implementation of the Fortran side too.

An example of that last point, which I'd like your thoughts on: I now think attempting a global set_xerbla is more trouble than it's worth. Making the tests use the set_* functions has shown that it can be valuable to update both the C and the Fortran handlers together on the BLAS side, and I don't want to get into the nightmarish combinatorial explosion of trying to achieve similar with an LAPACKE implementation that might-or-might-not install a handler for BLAS. Making separate calls to install a (C)BLAS and an LAPACK(E) handler is not onerous. There's also the factor that the CBLAS and LAPACKE xerbla situation are almost opposite in terms of capability, and certainly not very compatible with each other.

To outline the approach that I think will work for Windows: I'm going to try just conditionally (on Windows, and if it works also on macOS) not supplying, in CBLAS, the Fortran-compatible {set,get}_blas_xerbla. That means that when the CBLAS code wants to call those, they'll actually just be talking directly to BLAS, which will then be correctly influenced.

If that works, then I will add a CI job for macOS that doesn't add USE_FLAT_NAMESPACE, leaving the one that does since I don't want to force users to change their code if they're using linker stuff and it works for them. A benefit of having a non-flat alternative is that e.g. Homebrew are very reluctant to binary-distribute code with flat namespace, and given my reading of the Apple tea-leaves (edit: Homebrew/homebrew-core#300408), support for it is likely to disappear in time.

[mohawk2]

The errors being seen now (e.g. in https://github.com/Reference-LAPACK/lapack/actions/runs/39668775329) are of the nature:

Testing COMPLEX16 'Symmetric Eigenvalue Problem' (zsep_64.out) - passed: 78410
-->  *** XERBLA was called from ZHEEVD_2STAGE_64 with INFO =      8 instead of 10 ***
Info Error: 1

Testing COMPLEX16 'Symmetric Eigenvalue Problem 2-stage' (zse2_64.out) - passed: 77280
-->  *** XERBLA was called from ZHEEVD_2STAGE_64 with INFO =      8 instead of 10 ***
Info Error: 1

An attempt at isolating this, using the upterm technique mentioned in mpimd-csc/flexiblas#85 and these commands, having removed most CI jobs and changed the build type to Debug:

cd $GITHUB_WORKSPACE
ctest --parallel --build-dir build/ -R ^LAPACK-xeigtstz_se2_in --verbose
gdb build/bin/xeigtstz.exe
break ZHEEVD_2STAGE
run < TESTING/se2.in

The error occurs on the 9th entry to the routine, with stack-trace:

(gdb) bt
#0  zheevd_2stage (jobz=..., uplo=..., n=2, a=..., lda=2, w=..., work=..., 
    lwork=25, rwork=..., lrwork=1, iwork=..., liwork=1, info=-10, _jobz=1, 
    _uplo=1) at D:/a/lapack/lapack/SRC/zheevd_2stage.f:243
#1  0x00007ff66009e9fb in zerrst (path=..., nunit=6, _path=3)
    at D:/a/lapack/lapack/TESTING/EIG/zerrst.f:539
#2  0x00007ff65ffd9dc1 in zchkee ()
    at D:/a/lapack/lapack/TESTING/EIG/zchkee.F:1901

The test calls zheevd_2stage with an lrwork of 25, which is supposed to be enough - on most platforms the minimum is 23, as calculated from N + 1 + LHTRD + LWTRD. But on Windows, LWTRD is 642, which is greater than lrwork, so that parameter (the 8th) is considered invalid, and the 10th (also deliberately invalid) is never checked. LWTRD is calculated by this:

            LWTRD = ILAENV2STAGE( 4, 'ZHETRD_2STAGE', JOBZ='N',
     $                            N=2, KD=16, IB=16, -1 )

The fix I'm going with is to set the lwork value to a much higher value, since the calculated minimum value is dependent on threads anyway.