Tutorial
The package SolverTest provides a set a functions to test the basic features of a JSO-compliant solver. It is assumed that:
- the tested solver accepts an
AbstractNLPModelas input, and; - returns a
GenericExecutionStatsas output; - The solver function should accepts
atolandrtolas keyword arguments.
As shown in the examples below, the package also provide multiprecision tests for compatible solvers.
Unconstrained Optimization
The function unconstrained_nlp_set returns a list of optimization problems in the ADNLPModel format.
Secondly, the function unconstrained_nlp will test the behavior of the JSO-compliant function solver on the test set. More precisely, it will test that primal and dual feasibility at the solution (stats.solution) computed with kkt_checker and that stats.dual_feas are relatively small, i.e. ‖∇f(xᵏ)‖ ≤ atol + rtol * ‖∇f(x⁰)‖ where atol = rtol = 1e-6. It will also be tested that the final status stats.status is :first_order. The problems are sufficiently simple that we expect the solver to solve them successfully.
using SolverTest
using Test
using JSOSolvers # export solver trunk
@testset "Unconstrained solvers" begin
unconstrained_nlp(trunk)
multiprecision_nlp(trunk, :unc)
endTest.DefaultTestSet("Unconstrained solvers", Any[Test.DefaultTestSet("Problem (x₁ - 1)² + 4(x₂ - 1)²", Any[], 4, false, false, true, 1.748902411867771e9, 1.748902425716217e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Problem Diagonal quadratic", Any[], 4, false, false, true, 1.748902425716247e9, 1.748902426907424e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Problem Tridiagonal quadratic", Any[], 4, false, false, true, 1.748902426907453e9, 1.748902428236496e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Problem Rosenbrock", Any[], 4, false, false, true, 1.748902428236527e9, 1.748902428890416e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Problem Extended Rosenbrock", Any[], 4, false, false, true, 1.748902428890445e9, 1.748902429690295e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Precision Float16 for ptype unc", Any[], 7, false, false, true, 1.74890247430603e9, 1.748902497021374e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float32 for ptype unc", Any[], 7, false, false, true, 1.748902497021419e9, 1.748902518181132e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float64 for ptype unc", Any[], 7, false, false, true, 1.748902518181171e9, 1.748902519029141e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision BigFloat for ptype unc", Any[], 7, false, false, true, 1.748902519029188e9, 1.748902539104055e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl")], 0, false, false, true, 1.748902397717668e9, 1.748902539110529e9, false, "none")Similarly, unconstrained_nls_set and unconstrained_nls are preparing the same tests for nonlinear least squares.
using SolverTest
using Test
using JSOSolvers # export solver trunk
@testset "Unconstrained solvers" begin
unconstrained_nls(trunk)
multiprecision_nls(trunk, :unc)
endTest.DefaultTestSet("Unconstrained solvers", Any[Test.DefaultTestSet("Problem (x₁ - 1)² + 4(x₂ - 1)²", Any[], 4, false, false, true, 1.748902545639294e9, 1.748902548050721e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/unconstrained.jl"), Test.DefaultTestSet("Problem Diagonal quadratic", Any[], 4, false, false, true, 1.748902548050747e9, 1.748902548990426e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/unconstrained.jl"), Test.DefaultTestSet("Problem Non-zero residual", Any[], 4, false, false, true, 1.748902548990456e9, 1.748902549612826e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/unconstrained.jl"), Test.DefaultTestSet("Problem Rosenbrock", Any[], 4, false, false, true, 1.748902549612868e9, 1.748902550103593e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/unconstrained.jl"), Test.DefaultTestSet("Problem Overdetermined", Any[], 4, false, false, true, 1.74890255010362e9, 1.748902550685405e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/unconstrained.jl"), Test.DefaultTestSet("Problem Extended Rosenbrock", Any[], 4, false, false, true, 1.748902550685433e9, 1.748902551668328e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/unconstrained.jl"), Test.DefaultTestSet("Precision Float16 for ptype unc", Any[], 7, false, false, true, 1.748902552196215e9, 1.748902555741578e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision Float32 for ptype unc", Any[], 7, false, false, true, 1.748902555741625e9, 1.748902558964976e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision Float64 for ptype unc", Any[], 7, false, false, true, 1.748902558965023e9, 1.74890255974633e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision BigFloat for ptype unc", Any[], 7, false, false, true, 1.748902559746368e9, 1.748902563511495e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl")], 0, false, false, true, 1.748902539438955e9, 1.74890256351152e9, false, "none")Bound-constrained Optimization
The function bound_constrained_nlp_set returns a list of optimization problems in the ADNLPModel format.
Secondly, the function bound_constrained_nlp will test the behavior of the JSO-compliant function solver on the test set. More precisely, it will test that primal and dual feasibility at the solution (stats.solution) computed with kkt_checker and that stats.primal_feas and stats.dual_feas are relatively small, i.e. smaller than atol + rtol * ‖∇f(x⁰)‖ where atol = rtol = 1e-6. It will also be tested that the final status stats.status is :first_order. The problems are sufficiently simple that we expect the solver to solve them successfully.
using SolverTest
using Test
using JSOSolvers # export solver trunk
@testset "Bound-constrained solvers" begin
bound_constrained_nlp(tron)
multiprecision_nlp(tron, :unc)
multiprecision_nlp(tron, :bnd)
endTest.DefaultTestSet("Bound-constrained solvers", Any[Test.DefaultTestSet("Problem Simple quadratic", Any[], 4, false, false, true, 1.748902566680724e9, 1.748902568777074e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/bound-constrained.jl"), Test.DefaultTestSet("Problem Rosenbrock inactive bounds", Any[], 4, false, false, true, 1.748902568777106e9, 1.748902569414618e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/bound-constrained.jl"), Test.DefaultTestSet("Problem Rosenbrock active bounds", Any[], 4, false, false, true, 1.748902569414649e9, 1.748902570017555e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/bound-constrained.jl"), Test.DefaultTestSet("Problem One fixed variable", Any[], 4, false, false, true, 1.748902570017584e9, 1.748902570693584e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/bound-constrained.jl"), Test.DefaultTestSet("Problem All variables fixed", Any[], 4, false, false, true, 1.74890257069361e9, 1.748902571106911e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/bound-constrained.jl"), Test.DefaultTestSet("Problem Extended Rosenbrock", Any[], 4, false, false, true, 1.748902571106942e9, 1.748902571925207e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/bound-constrained.jl"), Test.DefaultTestSet("Precision Float16 for ptype unc", Any[], 7, false, false, true, 1.748902572453092e9, 1.748902574936371e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float32 for ptype unc", Any[], 7, false, false, true, 1.748902574936413e9, 1.74890257693933e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float64 for ptype unc", Any[], 7, false, false, true, 1.748902576939371e9, 1.748902577251734e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision BigFloat for ptype unc", Any[], 7, false, false, true, 1.748902577251769e9, 1.748902579473099e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float16 for ptype bnd", Any[], 7, false, false, true, 1.74890257947315e9, 1.74890257951169e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float32 for ptype bnd", Any[], 7, false, false, true, 1.748902579511717e9, 1.748902579549012e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float64 for ptype bnd", Any[], 7, false, false, true, 1.748902579549037e9, 1.748902579552427e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision BigFloat for ptype bnd", Any[], 7, false, false, true, 1.748902579552449e9, 1.748902579590871e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl")], 0, false, false, true, 1.748902563517341e9, 1.748902579590891e9, false, "none")Similarly, bound_constrained_nls_set and bound_constrained_nls are preparing the same tests for nonlinear least squares.
using SolverTest
using Test
using JSOSolvers # export solver trunk
@testset "Bound-constrained solvers" begin
bound_constrained_nls(tron)
multiprecision_nls(tron, :unc)
multiprecision_nls(tron, :bnd)
endTest.DefaultTestSet("Bound-constrained solvers", Any[Test.DefaultTestSet("Problem Simple quadratic", Any[], 4, false, false, true, 1.748902584703623e9, 1.748902586499086e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/bound-constrained.jl"), Test.DefaultTestSet("Problem Non-zero residual", Any[], 4, false, false, true, 1.748902586499115e9, 1.748902586993501e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/bound-constrained.jl"), Test.DefaultTestSet("Problem Rosenbrock inactive bounds", Any[], 4, false, false, true, 1.748902586993527e9, 1.748902587539082e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/bound-constrained.jl"), Test.DefaultTestSet("Problem Rosenbrock active bounds", Any[], 4, false, false, true, 1.74890258753911e9, 1.748902588040311e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/bound-constrained.jl"), Test.DefaultTestSet("Problem One fixed variable", Any[], 4, false, false, true, 1.748902588040338e9, 1.748902588550018e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/bound-constrained.jl"), Test.DefaultTestSet("Problem All variables fixed", Any[], 4, false, false, true, 1.748902588550043e9, 1.748902589240384e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/bound-constrained.jl"), Test.DefaultTestSet("Problem Extended Rosenbrock", Any[], 4, false, false, true, 1.748902589240418e9, 1.748902590044278e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/bound-constrained.jl"), Test.DefaultTestSet("Precision Float16 for ptype unc", Any[], 7, false, false, true, 1.748902590407625e9, 1.748902592295967e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision Float32 for ptype unc", Any[], 7, false, false, true, 1.748902592296009e9, 1.74890259391064e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision Float64 for ptype unc", Any[], 7, false, false, true, 1.74890259391068e9, 1.74890259405393e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision BigFloat for ptype unc", Any[], 7, false, false, true, 1.748902594053962e9, 1.748902595949794e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision Float16 for ptype bnd", Any[], 7, false, false, true, 1.748902595949862e9, 1.748902596661919e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision Float32 for ptype bnd", Any[], 7, false, false, true, 1.748902596661954e9, 1.748902597443073e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision Float64 for ptype bnd", Any[], 7, false, false, true, 1.748902597443111e9, 1.748902598175777e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl"), Test.DefaultTestSet("Precision BigFloat for ptype bnd", Any[], 7, false, false, true, 1.748902598175813e9, 1.748902599207933e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nls/multiprecision.jl")], 0, false, false, true, 1.748902579596568e9, 1.748902599207954e9, false, "none")Equality-constrained Optimization
This expands straightforwardly to equality-constrained problems.
using SolverTest
using Test
using DCISolver # export solver dci
@testset "Testing DCI-solver" begin
@testset "$foo" for foo in [
unconstrained_nlp,
equality_constrained_nlp,
]
foo(dci)
end
@testset "Multiprecision tests" begin
for ptype in [:unc, :equ]
multiprecision_nlp(
dci,
ptype,
precisions = (Float32, Float64),
)
end
end
endTest.DefaultTestSet("Testing DCI-solver", Any[Test.DefaultTestSet("unconstrained_nlp", Any[Test.DefaultTestSet("Problem (x₁ - 1)² + 4(x₂ - 1)²", Any[], 4, false, false, true, 1.748902601617492e9, 1.748902606663306e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Problem Diagonal quadratic", Any[], 4, false, false, true, 1.748902606663328e9, 1.748902608410626e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Problem Tridiagonal quadratic", Any[], 4, false, false, true, 1.74890260841065e9, 1.748902608980254e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Problem Rosenbrock", Any[], 4, false, false, true, 1.748902608980277e9, 1.748902608997387e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl"), Test.DefaultTestSet("Problem Extended Rosenbrock", Any[], 4, false, false, true, 1.748902608997407e9, 1.748902609231403e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/unconstrained.jl")], 0, false, false, true, 1.748902600655171e9, 1.748902609231408e9, false, "none"), Test.DefaultTestSet("equality_constrained_nlp", Any[Test.DefaultTestSet("Problem Simple quadratic problem", Any[], 5, false, false, true, 1.748902611620336e9, 1.74890261516934e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/equality-constrained.jl"), Test.DefaultTestSet("Problem HS6", Any[], 5, false, false, true, 1.748902615169372e9, 1.748902615972266e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/equality-constrained.jl"), Test.DefaultTestSet("Problem scaled HS8", Any[], 5, false, false, true, 1.748902615972288e9, 1.748902616760093e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/equality-constrained.jl"), Test.DefaultTestSet("Problem ‖x‖² s.t. ∑x = n", Any[], 5, false, false, true, 1.748902616760116e9, 1.748902618168961e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/equality-constrained.jl"), Test.DefaultTestSet("Problem Rosenbrock with ∑x = 2", Any[], 5, false, false, true, 1.748902618168993e9, 1.748902619125583e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/equality-constrained.jl")], 0, false, false, true, 1.748902609231445e9, 1.748902619125587e9, false, "none"), Test.DefaultTestSet("Multiprecision tests", Any[Test.DefaultTestSet("Precision Float32 for ptype unc", Any[], 7, false, false, true, 1.748902619125636e9, 1.7489026244894e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float64 for ptype unc", Any[], 7, false, false, true, 1.748902624489434e9, 1.748902625241644e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float32 for ptype equ", Any[], 7, false, false, true, 1.748902625241678e9, 1.748902629284789e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl"), Test.DefaultTestSet("Precision Float64 for ptype equ", Any[], 7, false, false, true, 1.748902629284828e9, 1.748902629967965e9, false, "/home/runner/work/SolverTest.jl/SolverTest.jl/src/nlp/multiprecision.jl")], 0, false, false, true, 1.74890261912561e9, 1.748902629967969e9, false, "none")], 0, false, false, true, 1.748902600655075e9, 1.74890262996797e9, false, "none")Bound and Equality-constrained Optimization
This is still work in progress, see Issue 8.
Inequality-constrained optimization
This is still work in progress, see Issue 8.