Re-solve and in-place solve

It is very convenient to pre-allocate the memory used during the optimization of a given problem either for improved memory management or re-solving the same or a similar problem.

Let us consider the following 2-dimensional unconstrained problem

\[\begin{aligned} \min_x \quad & f(x):= x_2^2 \exp(x_1^2) \end{aligned}\]

Using JSOSuite’s minimize function, the problem can be solved as follows

using JSOSuite
f(x) = x[2]^2 * exp(x[1]^2)
stats = minimize(f, ones(2))
;

The problem has 2 variables and no constraints.
Algorithm selection:
- unconstrained: ✓;
- nonlinear objective: ✓;
- may use 2-th order derivative.
There are 6 optimizers available:
["LBFGS", "R2", "TRON", "TRUNK", "IPOPT", "Percival"].
Solve using LBFGS:
[ Info:   iter      f(x)      ‖∇f‖      ∇fᵀd      bk
[ Info:      0   2.7e+00   7.7e+00         -       -
[ Info:      1   1.7e-02   2.6e-01  -5.9e+01       2
[ Info:      2   7.5e-03   1.7e-01  -1.2e-02       0
[ Info:      3   2.1e-09   9.3e-05  -1.5e-02       0
[ Info:      4   2.3e-17   9.6e-09  -4.3e-09       0
[ Info:      4   2.3e-17   9.6e-09

Using L-BFGS, the problem is locally solved.

Note that when passing Julia functions as input to minimize, the problem is modeled as an ADNLPModel. So, the following would be equivalent:

using ADNLPModels, JSOSuite
f(x) = x[2]^2 * exp(x[1]^2)
nlp = ADNLPModel(f, ones(2))
stats = minimize(nlp)

"Execution stats: first-order stationary"

The procedure is similar with JuMP models.

using JuMP, JSOSuite
model = Model()
@variable(model, x[i=1:2], start = [1.0; 1.0][i])
@objective(model, Min, x[2]^2 * exp(x[1]^2))
stats = minimize(model)

"Execution stats: first-order stationary"

In-place solve

If we want to solve the same problem several times, for instance, for several initial guesses, it is recommended to use an in-place solve.

using ADNLPModels, JSOSolvers, SolverCore
f(x) = x[2]^2 * exp(x[1]^2)
nlp = ADNLPModel(f, ones(2)) # or use JuMP
solver = JSOSolvers.LBFGSSolver(nlp)
stats = SolverCore.GenericExecutionStats(nlp)
solve!(solver, nlp, stats, x = ones(2))

"Execution stats: first-order stationary"

This deserves more explanations. The name of the solver structure and the corresponding package can be accessed via the DataFrame JSOSuite.optimizers.

JSOSuite.optimizers[!, [:name_solver, :name_pkg]]

In our example, the solver L-BFGS is implemented in JSOSolvers.jl and the solver structure is LBFGSSolver.

Now, it is possible to reuse the memory allocated for the first solve for another round:

# NLPModels.reset!(nlp) # would also reset the evaluation counters of the model
SolverCore.reset!(solver)
new_x0 = [1.4; 5.0] # another initial guess
solve!(solver, nlp, stats, x = new_x0)  # new solve with existing solver object

"Execution stats: first-order stationary"

In-place solve of a different problem

It is also possible to reuse the allocated memory to solve another problem with the same number of variables and constraints:

f2(x) = x[2]^2 + exp(x[1]^2)
nlp = ADNLPModel(f2, ones(2)) # or use JuMP
SolverCore.reset!(solver, nlp)
solve!(solver, nlp, stats)

"Execution stats: first-order stationary"

Allocation-free solvers

In order to measure, the amount of memory allocated by the solvers, the package NLPModelsTest.jl defines a set of test problems that are allocation free.

using NLPModelsTest, SolverCore, JSOSolvers
nlp = BROWNDEN(Float64)
solver = LBFGSSolver(nlp)
stats = GenericExecutionStats(nlp)
solve!(solver, nlp, stats)
@allocated solve!(solver, nlp, stats)

0

Several of the pure Julia solvers available in JSOSuite have this property.