diff --git a/previews/PR134/example.ipynb b/previews/PR134/example.ipynb index d089189..9ca1c3a 100644 --- a/previews/PR134/example.ipynb +++ b/previews/PR134/example.ipynb @@ -467,7 +467,7 @@ { "output_type": "execute_result", "data": { - "text/plain": "(13.891, 14.632610082626343)" + "text/plain": "(13.915, 14.954005002975464)" }, "metadata": {}, "execution_count": 16 diff --git a/previews/PR134/example/index.html b/previews/PR134/example/index.html index 52f1adf..0f462bd 100644 --- a/previews/PR134/example/index.html +++ b/previews/PR134/example/index.html @@ -60,5 +60,5 @@ jprod_nln: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jtprod: ██████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5 jtprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jtprod_nln: ██████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5 hess: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 hprod: ████████████████████ 10 jhess: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jhprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 -

We now compare the two solvers with respect to the time spent,

stats_ipopt.elapsed_time, stats_fps_solve.elapsed_time
(17.899, 23.588104963302612)

and also check objective value, feasibility and dual feasibility of ipopt and fps_solve.

(stats_ipopt.objective, stats_ipopt.primal_feas, stats_ipopt.dual_feas),
-(stats_fps_solve.objective, stats_fps_solve.primal_feas, stats_fps_solve.dual_feas)
((0.005425026428348349, 2.2204460492503135e-18, 4.2724309076258577e-7), (0.005425025968573663, 2.2204460492503135e-18, 2.968652346957424e-7))

Overall FletcherPenaltySolver is doing great for solving large-scale optimization problems!

This page was generated using Literate.jl.

+

We now compare the two solvers with respect to the time spent,

stats_ipopt.elapsed_time, stats_fps_solve.elapsed_time
(17.819, 23.132580041885376)

and also check objective value, feasibility and dual feasibility of ipopt and fps_solve.

(stats_ipopt.objective, stats_ipopt.primal_feas, stats_ipopt.dual_feas),
+(stats_fps_solve.objective, stats_fps_solve.primal_feas, stats_fps_solve.dual_feas)
((0.005425026428348349, 2.2204460492503135e-18, 4.2724309076258577e-7), (0.005425025968573663, 2.2204460492503135e-18, 2.968652346957424e-7))

Overall FletcherPenaltySolver is doing great for solving large-scale optimization problems!

This page was generated using Literate.jl.

diff --git a/previews/PR134/fine-tuneFPS/index.html b/previews/PR134/fine-tuneFPS/index.html index c1bddf3..095e5cf 100644 --- a/previews/PR134/fine-tuneFPS/index.html +++ b/previews/PR134/fine-tuneFPS/index.html @@ -18,4 +18,4 @@ stp = NLPStopping(nlp) data = FPSSSolver(stp) stats = GenericExecutionStats(nlp) -stats = SolverCore.solve!(data, stp, stats)
"Execution stats: first-order stationary"

List of possible options

Find below a list of the main options of fps_solve.

Tolerances on the problem

We use Stopping.jl to control the algorithmic flow, we refer to Stopping.jl and https://solverstoppingjulia.github.io for tutorials and documentation. By default, we use the function Fletcher_penalty_optimality_check as optimality check, and the default tol_check is rtol [1 + c(x₀); 1 + ∇f(x₀)] with rtol = 1e-7.

Additional parameters used in stopping the algorithm are defined in the following table.

ParametersTypeDefaultDescription
lagrange_boundReal1 / sqrt(eps(T))bounds on estimated Lagrange multipliers.
subsolver_max_iterReal20000maximum iteration for the subproblem solver.

Algorithmic parameters

The metadata is defined in a AlgoData structure at the initialization of FPSSolver.

ParametersTypeDefaultDescription
σ_0Real1e3Initialize subproblem's parameter σ
σ_maxReal1 / √eps(T)Maximum value for subproblem's parameter σ
σ_updateRealT(2)Update subproblem's parameter σ
ρ_0RealT(2)Initialize subproblem's parameter ρ
ρ_maxReal1 / √eps(T)Maximum value for subproblem's parameter ρ
ρ_updateRealT(2)Update subproblem's parameter ρ
δ_0Real√eps(T)Initialize subproblem's parameter δ
δ_maxReal1 / √eps(T)Maximum value for subproblem's parameter δ
δ_updateRealT(10)Update subproblem's parameter δ
η_1Realzero(T)Initialize subproblem's parameter η
η_updateRealone(T)Update subproblem's parameter η
yMRealtypemax(T)bound on the Lagrange multipliers
ΔRealT(0.95)expected decrease in feasibility between two iterations
subproblem_solverFunctionKNITRO.has_knitro() ? NLPModelsKnitro.knitro : ipoptsolver used for the subproblem, see also JSOSolvers.jl
subpb_unbounded_thresholdReal1 / √eps(T)below the opposite of this value, the subproblem is unbounded
atol_subFunctionatol -> atolabsolute tolerance for the subproblem in function of atol
rtol_subFunctionrtol -> rtolrelative tolerance for the subproblem in function of rtol
hessian_approxeither Val(1) or Val(2)Val(2)it selects the hessian approximation
convex_subproblemBoolfalsetrue if the subproblem is convex. Useful to set the convex option in knitro.
qds_solverSymbol:ldltInitialize the QDSolver to solve quasi-definite systems, either :ldlt or :iterative.

Feasibility step

The metadata for the feasibility procedure is defined in a GNSolver structure at the initialization of FPSSolver.

ParametersTypeDefaultDescription
η₁Real1e-3Feasibility step: decrease the trust-region radius when Ared/Pred < η₁.
η₂Real0.66Feasibility step: increase the trust-region radius when Ared/Pred > η₂.
σ₁Real0.25Feasibility step: decrease coefficient of the trust-region radius.
σ₂Real2.0Feasibility step: increase coefficient of the trust-region radius.
Δ₀Real1.0Feasibility step: initial radius.
feas_expected_decreaseReal0.95Feasibility step: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feasexpecteddecrease.
bad_steps_limInteger3Feasibility step: consecutive bad steps before using a second order step.
TR_compute_stepKrylovSolverLsmrSolverCompute the direction in feasibility step.
aggressive_stepKrylovSolverCgSolverCompute the (aggressive) direction in feasibility step.
+stats = SolverCore.solve!(data, stp, stats)
"Execution stats: first-order stationary"

List of possible options

Find below a list of the main options of fps_solve.

Tolerances on the problem

We use Stopping.jl to control the algorithmic flow, we refer to Stopping.jl and https://solverstoppingjulia.github.io for tutorials and documentation. By default, we use the function Fletcher_penalty_optimality_check as optimality check, and the default tol_check is rtol [1 + c(x₀); 1 + ∇f(x₀)] with rtol = 1e-7.

Additional parameters used in stopping the algorithm are defined in the following table.

ParametersTypeDefaultDescription
lagrange_boundReal1 / sqrt(eps(T))bounds on estimated Lagrange multipliers.
subsolver_max_iterReal20000maximum iteration for the subproblem solver.

Algorithmic parameters

The metadata is defined in a AlgoData structure at the initialization of FPSSolver.

ParametersTypeDefaultDescription
σ_0Real1e3Initialize subproblem's parameter σ
σ_maxReal1 / √eps(T)Maximum value for subproblem's parameter σ
σ_updateRealT(2)Update subproblem's parameter σ
ρ_0RealT(2)Initialize subproblem's parameter ρ
ρ_maxReal1 / √eps(T)Maximum value for subproblem's parameter ρ
ρ_updateRealT(2)Update subproblem's parameter ρ
δ_0Real√eps(T)Initialize subproblem's parameter δ
δ_maxReal1 / √eps(T)Maximum value for subproblem's parameter δ
δ_updateRealT(10)Update subproblem's parameter δ
η_1Realzero(T)Initialize subproblem's parameter η
η_updateRealone(T)Update subproblem's parameter η
yMRealtypemax(T)bound on the Lagrange multipliers
ΔRealT(0.95)expected decrease in feasibility between two iterations
subproblem_solverFunctionKNITRO.has_knitro() ? NLPModelsKnitro.knitro : ipoptsolver used for the subproblem, see also JSOSolvers.jl
subpb_unbounded_thresholdReal1 / √eps(T)below the opposite of this value, the subproblem is unbounded
atol_subFunctionatol -> atolabsolute tolerance for the subproblem in function of atol
rtol_subFunctionrtol -> rtolrelative tolerance for the subproblem in function of rtol
hessian_approxeither Val(1) or Val(2)Val(2)it selects the hessian approximation
convex_subproblemBoolfalsetrue if the subproblem is convex. Useful to set the convex option in knitro.
qds_solverSymbol:ldltInitialize the QDSolver to solve quasi-definite systems, either :ldlt or :iterative.

Feasibility step

The metadata for the feasibility procedure is defined in a GNSolver structure at the initialization of FPSSolver.

ParametersTypeDefaultDescription
η₁Real1e-3Feasibility step: decrease the trust-region radius when Ared/Pred < η₁.
η₂Real0.66Feasibility step: increase the trust-region radius when Ared/Pred > η₂.
σ₁Real0.25Feasibility step: decrease coefficient of the trust-region radius.
σ₂Real2.0Feasibility step: increase coefficient of the trust-region radius.
Δ₀Real1.0Feasibility step: initial radius.
feas_expected_decreaseReal0.95Feasibility step: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feasexpecteddecrease.
bad_steps_limInteger3Feasibility step: consecutive bad steps before using a second order step.
TR_compute_stepKrylovSolverLsmrSolverCompute the direction in feasibility step.
aggressive_stepKrylovSolverCgSolverCompute the (aggressive) direction in feasibility step.
diff --git a/previews/PR134/index.html b/previews/PR134/index.html index c4d3b23..f39c4f2 100644 --- a/previews/PR134/index.html +++ b/previews/PR134/index.html @@ -15,4 +15,4 @@ [0.0], [1.0], ) -stats = fps_solve(nlp)
"Execution stats: first-order stationary"

Bug reports and discussions

If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.

If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers, so questions about any of our packages are welcome.

+stats = fps_solve(nlp)
"Execution stats: first-order stationary"

Bug reports and discussions

If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.

If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers, so questions about any of our packages are welcome.

diff --git a/previews/PR134/reference/index.html b/previews/PR134/reference/index.html index a2f6c3d..ed044c2 100644 --- a/previews/PR134/reference/index.html +++ b/previews/PR134/reference/index.html @@ -1,19 +1,19 @@ Reference · FletcherPenaltySolver
Edit on GitHub

Reference

​

Contents

​

​

Index

​

​

FletcherPenaltySolver.AlgoData — Type
AlgoData(; kwargs...) 
-AlgoData(T::DataType; kwargs...)

Structure containing all the parameters used in the fps_solve call. T is the datatype used in the algorithm, by default it is Float64. Returns a AlgoData structure.

Arguments

The keyword arguments may include:

  • σ_0::Real = T(1e3): Initialize subproblem's parameter σ;
  • σ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter σ;
  • σ_update::Real = T(2): Update subproblem's parameter σ;
  • ρ_0::Real = one(T): Initialize subproblem's parameter ρ;
  • ρ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter ρ;
  • ρ_update::Real = T(2): Update subproblem's parameter ρ;
  • δ_0::Real = √eps(T): Initialize subproblem's parameter δ;
  • δ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter δ;
  • δ_update::Real = T(10): Update subproblem's parameter δ;
  • η_1::Real = zero(T): Initialize subproblem's parameter η;
  • η_update::Real = one(T): Update subproblem's parameter η;
  • yM::Real = typemax(T): bound on the Lagrange multipliers;
  • Δ::Real = T(0.95): expected decrease in feasibility between two iterations;
  • subproblem_solver::Function = ipopt: solver used for the subproblem;
  • subpb_unbounded_threshold::Real = 1 / √eps(T): below the opposite of this value, the subproblem is unbounded;
  • subsolver_max_iter::Int = 20000: maximum of iteration for the subproblem solver;
  • atol_sub::Function = atol -> atol: absolute tolerance for the subproblem in function of atol;
  • rtol_sub::Function = rtol -> rtol: relative tolerance for the subproblem in function of rtol;
  • hessian_approx = Val(2): either Val(1) or Val(2), it selects the hessian approximation;
  • convex_subproblem::Bool = false: true if the subproblem is convex. Useful to set the convex option in knitro;
  • lagrange_bound::T = 1 / sqrt(eps(T)): upper-bound on the Lagrange multiplier.

For more details, we refer to the package documentation fine-tuneFPS.md.

source
FletcherPenaltySolver.FPSSSolver — Type
FPSSSolver(nlp::AbstractNLPModel [, x0 = nlp.meta.x0]; kwargs...)
-FPSSSolver(stp::NLPStopping; kwargs...)

Structure regrouping all the structure used during the fps_solve call. It returns a FPSSSolver structure.

Arguments

The keyword arguments may include:

  • stp::NLPStopping: Stopping structure for this algorithm workflow;
  • meta::AlgoData{T}: see AlgoData;
  • workspace: allocated space for the solver itself;
  • qdsolver: solver structure for the linear algebra part, contains allocation for this part. By default a LDLtSolver, but an alternative is IterativeSolver ;
  • subproblem_solver::AbstractOptimizationSolver: by default a subproblem_solver_correspondence[Symbol(meta.subproblem_solver)];
  • sub_stats::GenericExecutionStats: stats structure for the result of subproblem_solver;
  • feasibility_solver: by default a GNSolver, see GNSolver;
  • model::FletcherPenaltyNLP: subproblem;
  • sub_stp::NLPStopping: Stopping structure for the subproblem.

Note:

  • subproblem_solver is accessible from the subproblem_solver_correspondence::Dict.
  • the qdsolver is accessible from the dictionary qdsolver_correspondence.
source
FletcherPenaltySolver.FletcherPenaltyNLP — Type
FletcherPenaltyNLP(nlp, σ, hessian_approx, [x0 = nlp.meta.x0]; qds = LDLtSolver(nlp, S(0)))
+AlgoData(T::DataType; kwargs...)

Structure containing all the parameters used in the fps_solve call. T is the datatype used in the algorithm, by default it is Float64. Returns a AlgoData structure.

Arguments

The keyword arguments may include:

  • σ_0::Real = T(1e3): Initialize subproblem's parameter σ;
  • σ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter σ;
  • σ_update::Real = T(2): Update subproblem's parameter σ;
  • ρ_0::Real = one(T): Initialize subproblem's parameter ρ;
  • ρ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter ρ;
  • ρ_update::Real = T(2): Update subproblem's parameter ρ;
  • δ_0::Real = √eps(T): Initialize subproblem's parameter δ;
  • δ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter δ;
  • δ_update::Real = T(10): Update subproblem's parameter δ;
  • η_1::Real = zero(T): Initialize subproblem's parameter η;
  • η_update::Real = one(T): Update subproblem's parameter η;
  • yM::Real = typemax(T): bound on the Lagrange multipliers;
  • Δ::Real = T(0.95): expected decrease in feasibility between two iterations;
  • subproblem_solver::Function = ipopt: solver used for the subproblem;
  • subpb_unbounded_threshold::Real = 1 / √eps(T): below the opposite of this value, the subproblem is unbounded;
  • subsolver_max_iter::Int = 20000: maximum of iteration for the subproblem solver;
  • atol_sub::Function = atol -> atol: absolute tolerance for the subproblem in function of atol;
  • rtol_sub::Function = rtol -> rtol: relative tolerance for the subproblem in function of rtol;
  • hessian_approx = Val(2): either Val(1) or Val(2), it selects the hessian approximation;
  • convex_subproblem::Bool = false: true if the subproblem is convex. Useful to set the convex option in knitro;
  • lagrange_bound::T = 1 / sqrt(eps(T)): upper-bound on the Lagrange multiplier.

For more details, we refer to the package documentation fine-tuneFPS.md.

source
FletcherPenaltySolver.FPSSSolver — Type
FPSSSolver(nlp::AbstractNLPModel [, x0 = nlp.meta.x0]; kwargs...)
+FPSSSolver(stp::NLPStopping; kwargs...)

Structure regrouping all the structure used during the fps_solve call. It returns a FPSSSolver structure.

Arguments

The keyword arguments may include:

  • stp::NLPStopping: Stopping structure for this algorithm workflow;
  • meta::AlgoData{T}: see AlgoData;
  • workspace: allocated space for the solver itself;
  • qdsolver: solver structure for the linear algebra part, contains allocation for this part. By default a LDLtSolver, but an alternative is IterativeSolver ;
  • subproblem_solver::AbstractOptimizationSolver: by default a subproblem_solver_correspondence[Symbol(meta.subproblem_solver)];
  • sub_stats::GenericExecutionStats: stats structure for the result of subproblem_solver;
  • feasibility_solver: by default a GNSolver, see GNSolver;
  • model::FletcherPenaltyNLP: subproblem;
  • sub_stp::NLPStopping: Stopping structure for the subproblem.

Note:

  • subproblem_solver is accessible from the subproblem_solver_correspondence::Dict.
  • the qdsolver is accessible from the dictionary qdsolver_correspondence.
source
FletcherPenaltySolver.FletcherPenaltyNLP — Type
FletcherPenaltyNLP(nlp, σ, hessian_approx, [x0 = nlp.meta.x0]; qds = LDLtSolver(nlp, S(0)))
 FletcherPenaltyNLP(nlp, σ, ρ, δ, hessian_approx, [x0 = nlp.meta.x0]; qds = LDLtSolver(nlp, S(0)))
 FletcherPenaltyNLP(nlp; σ_0 = 1, ρ_0 = 0, δ_0 = 0, hessian_approx = Val(2), x0 = nlp.meta.x0, qds = LDLtSolver(nlp, S(0)))

We consider here the implementation of Fletcher's exact penalty method for the minimization problem:

\[ minₓ f(x) s.t. c(x) = ℓ, l ≤ x ≤ u\]

using Fletcher penalty function:

\[ minₓ f(x) - (c(x) - ℓ)^T ys(x) + 0.5 ρ ||c(x) - ℓ||²₂ s.t. l ≤ x ≤ u\]

where

\[ ys(x) ∈ arg min\_y 0.5 ||A(x)y - g(x)||²₂ + σ (c(x) - ℓ)^T y + 0.5 || δ ||²₂\]

Arguments

  • nlp::AbstractNLPModel: the model solved, see NLPModels.jl;
  • x::AbstractVector: Initial guess. If x is not specified, then nlp.meta.x0 is used;
  • σ, ρ, δ parameters of the subproblem;
  • hessian_approx either Val(1) or Val(2) for the hessian approximation.
  • qds: solver structure for the linear algebra computations, see LDLtSolver or IterativeSolver.

Notes:

  • Evaluation of the obj, grad, objgrad functions evaluate functions from the orginial nlp. These values are stored in fx, cx, gx.
  • The value of the penalty vector ys is also stored.
  • The hessian's structure is dense.

Examples

julia> using FletcherPenaltySolver, ADNLPModels
 julia> nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0])
-julia> fp_sos  = FletcherPenaltyNLP(nlp)
source
FletcherPenaltySolver.GNSolver — Type
GNSolver(x, y; kwargs...)

Structure containing all the parameters used in the feasibility step. x is an intial guess, and y is an initial guess for the Lagrange multiplier. Returns a GNSolver structure.

Arguments

The keyword arguments may include:

  • η₁::T=T(1e-3): Feasibility step: decrease the trust-region radius when Ared/Pred < η₁.
  • η₂::T=T(0.66): Feasibility step: increase the trust-region radius when Ared/Pred > η₂.
  • σ₁::T=T(0.25): Feasibility step: decrease coefficient of the trust-region radius.
  • σ₂::T=T(2.0): Feasibility step: increase coefficient of the trust-region radius.
  • Δ₀::T=one(T): Feasibility step: initial radius.
  • bad_steps_lim::Integer=3: Feasibility step: consecutive bad steps before using a second order step.
  • feas_expected_decrease::T=T(0.95): Feasibility step: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feas_expected_decrease.
  • TR_compute_step = LsmrSolver(length(y0), length(x0), S): Compute the direction in feasibility step.
  • aggressive_step = CgSolver(length(x0), length(x0), S): Compute the direction in feasibility step in agressive mode.
source
FletcherPenaltySolver.Fletcher_penalty_optimality_check — Method
Fletcher_penalty_optimality_check(pb::AbstractNLPModel, state::NLPAtX)

Optimality function used by default in the algorithm. An alternative is to use the function KKT from Stopping.jl.

The function returns a vector of length ncon + nvar containing:

  • |c(x) - lcon| / |x|₂
  • res / |λ|₂ ; x - max(min(x - res, uvar), lvar)) if it has bounds

The fields x, cx and res need to be filled. If state.lambda is nothing then we take |λ|₂=1.

source
FletcherPenaltySolver.TR_lsmr — Method
TR_lsmr(solver, cz, Jz, ctol, Δ, normcz, Jd)

Compute a direction d such that

\[\begin{aligned} +julia> fp_sos = FletcherPenaltyNLP(nlp)

source
FletcherPenaltySolver.GNSolver — Type
GNSolver(x, y; kwargs...)

Structure containing all the parameters used in the feasibility step. x is an intial guess, and y is an initial guess for the Lagrange multiplier. Returns a GNSolver structure.

Arguments

The keyword arguments may include:

  • η₁::T=T(1e-3): Feasibility step: decrease the trust-region radius when Ared/Pred < η₁.
  • η₂::T=T(0.66): Feasibility step: increase the trust-region radius when Ared/Pred > η₂.
  • σ₁::T=T(0.25): Feasibility step: decrease coefficient of the trust-region radius.
  • σ₂::T=T(2.0): Feasibility step: increase coefficient of the trust-region radius.
  • Δ₀::T=one(T): Feasibility step: initial radius.
  • bad_steps_lim::Integer=3: Feasibility step: consecutive bad steps before using a second order step.
  • feas_expected_decrease::T=T(0.95): Feasibility step: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feas_expected_decrease.
  • TR_compute_step = LsmrSolver(length(y0), length(x0), S): Compute the direction in feasibility step.
  • aggressive_step = CgSolver(length(x0), length(x0), S): Compute the direction in feasibility step in agressive mode.
source
FletcherPenaltySolver.Fletcher_penalty_optimality_check — Method
Fletcher_penalty_optimality_check(pb::AbstractNLPModel, state::NLPAtX)

Optimality function used by default in the algorithm. An alternative is to use the function KKT from Stopping.jl.

The function returns a vector of length ncon + nvar containing:

  • |c(x) - lcon| / |x|₂
  • res / |λ|₂ ; x - max(min(x - res, uvar), lvar)) if it has bounds

The fields x, cx and res need to be filled. If state.lambda is nothing then we take |λ|₂=1.

source
FletcherPenaltySolver.TR_lsmr — Method
TR_lsmr(solver, cz, Jz, ctol, Δ, normcz, Jd)

Compute a direction d such that

\[\begin{aligned} \min_{d} \quad & \|c + Jz' d \| \\ \text{s.t.} \quad & \|d\| \leq \Delta, -\end{aligned}\]

using lsmr method from Krylov.jl.

Output

  • d: solution
  • Jd: product of the solution with J.
  • infeasible: true if the problem is infeasible.
  • solved: true if the problem has been successfully solved.
source
FletcherPenaltySolver.cons_norhs! — Method
cons_norhs!(nlp::FletcherPenaltyNLP, x, cx)

Redefine the NLPModel function cons! to account for non-zero right-hand side in the equation constraint. It returns cons(nlp, x) - nlp.meta.lcon.

source
FletcherPenaltySolver.feasibility_step — Method
feasibility_step(feasibility_solver, nlp, x, cx, normcx, Jx, ρ, ctol; kwargs...)

Approximately solves min ‖c(x) - l‖, where l is nlp.meta.lcon, using a trust-region Levenberg-Marquardt method.

Arguments

  • η₁::AbstractFloat = feasibility_solver.feas_η₁: decrease the trust-region radius when Ared/Pred < η₁.
  • η₂::AbstractFloat = feasibility_solver.feas_η₂: increase the trust-region radius when Ared/Pred > η₂.
  • σ₁::AbstractFloat = feasibility_solver.feas_σ₁: decrease coefficient of the trust-region radius.
  • σ₂::AbstractFloat = feasibility_solver.feas_σ₂:increase coefficient of the trust-region radius.
  • Δ₀::T = feasibility_solver.feas_Δ₀: initial trust-region radius.
  • bad_steps_lim::Integer = feasibility_solver.bad_steps_lim: consecutive bad steps before using a second order step.
  • expected_decrease::T = feasibility_solver.feas_expected_decrease: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feas_expected_decrease.
  • max_eval::Int = 1_000: maximum evaluations.
  • max_time::AbstractFloat = 60.0: maximum time.
  • max_feas_iter::Int = typemax(Int64): maximum number of iterations.

Output

  • z, cz, normcz, Jz: the new iterate, and updated evaluations.
  • status: Computation status. Possible outcomes are: :success, max_eval, max_time, max_iter, unknown_tired, :infeasible, :unknown.
source
FletcherPenaltySolver.fps_solve — Method
fps_solve(nlp::AbstractNLPModel, x0::AbstractVector{T} = nlp.meta.x0; subsolver_verbose::Int = 0, kwargs...)

Compute a local minimum of a bound and equality-constrained optimization problem using Fletcher's penalty function and the implementation described in

Estrin, R., Friedlander, M. P., Orban, D., & Saunders, M. A. (2020).
+\end{aligned}\]
using lsmr method from Krylov.jl.
Output
  • d: solution
  • Jd: product of the solution with J.
  • infeasible: true if the problem is infeasible.
  • solved: true if the problem has been successfully solved.
source
FletcherPenaltySolver.cons_norhs! — Method
cons_norhs!(nlp::FletcherPenaltyNLP, x, cx)

Redefine the NLPModel function cons! to account for non-zero right-hand side in the equation constraint. It returns cons(nlp, x) - nlp.meta.lcon.

source
FletcherPenaltySolver.feasibility_step — Method
feasibility_step(feasibility_solver, nlp, x, cx, normcx, Jx, ρ, ctol; kwargs...)

Approximately solves min ‖c(x) - l‖, where l is nlp.meta.lcon, using a trust-region Levenberg-Marquardt method.

Arguments

  • η₁::AbstractFloat = feasibility_solver.feas_η₁: decrease the trust-region radius when Ared/Pred < η₁.
  • η₂::AbstractFloat = feasibility_solver.feas_η₂: increase the trust-region radius when Ared/Pred > η₂.
  • σ₁::AbstractFloat = feasibility_solver.feas_σ₁: decrease coefficient of the trust-region radius.
  • σ₂::AbstractFloat = feasibility_solver.feas_σ₂:increase coefficient of the trust-region radius.
  • Δ₀::T = feasibility_solver.feas_Δ₀: initial trust-region radius.
  • bad_steps_lim::Integer = feasibility_solver.bad_steps_lim: consecutive bad steps before using a second order step.
  • expected_decrease::T = feasibility_solver.feas_expected_decrease: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feas_expected_decrease.
  • max_eval::Int = 1_000: maximum evaluations.
  • max_time::AbstractFloat = 60.0: maximum time.
  • max_feas_iter::Int = typemax(Int64): maximum number of iterations.

Output

  • z, cz, normcz, Jz: the new iterate, and updated evaluations.
  • status: Computation status. Possible outcomes are: :success, max_eval, max_time, max_iter, unknown_tired, :infeasible, :unknown.
source
FletcherPenaltySolver.fps_solve — Method
fps_solve(nlp::AbstractNLPModel, x0::AbstractVector{T} = nlp.meta.x0; subsolver_verbose::Int = 0, kwargs...)

Compute a local minimum of a bound and equality-constrained optimization problem using Fletcher's penalty function and the implementation described in

Estrin, R., Friedlander, M. P., Orban, D., & Saunders, M. A. (2020).
 Implementing a smooth exact penalty function for equality-constrained nonlinear optimization.
 SIAM Journal on Scientific Computing, 42(3), A1809-A1835.
 https://doi.org/10.1137/19M1238265

For advanced usage, the principal call to the solver uses a NLPStopping, see Stopping.jl.

fps_solve(stp::NLPStopping, fpssolver::FPSSSolver{T, QDS, US}; subsolver_verbose::Int = 0)
 fps_solve(stp::NLPStopping; subsolver_verbose::Int = 0, kwargs...)

Arguments

  • nlp::AbstractNLPModel: the model solved, see NLPModels.jl;
  • x: Initial guess. If x is not specified, then nlp.meta.x0 is used.

Keyword arguments

  • fpssolver: see FPSSSolver;
  • verbose::Int = 0: if > 0, display iteration information of the solver;
  • subsolver_verbose::Int = 0: if > 0, display iteration information of the subsolver;

All the information regarding stopping criteria can be set in the NLPStopping object. Additional kwargs are given to the NLPStopping. By default, the optimality condition used to declare optimality is Fletcher_penalty_optimality_check.

Output

The returned value is a GenericExecutionStats, see SolverCore.jl.

If one define a Stopping before calling fps_solve, it is possible to access all the information computed by the algorithm.

Notes

  • If the problem has inequalities, we use slack variables to get only equalities and bounds via NLPModelsModifiers.jl.
  • stp.current_state.res contains the gradient of Fletcher's penalty function.
  • subproblem_solver must take an NLPStopping as input, see StoppingInterface.jl.

Callback

The callback is called at each iteration. The expected signature of the callback is callback(nlp, solver, stats), and its output is ignored. Changing any of the input arguments will affect the subsequent iterations. In particular, setting stats.status = :user will stop the algorithm. All relevant information should be available in nlp and solver. Notably, you can access, and modify, the following:

  • solver: see FPSSSolver
  • stats: structure holding the output of the algorithm (GenericExecutionStats), which contains, among other things:
    • stats.dual_feas: norm of current gradient of the Lagrangian;
    • stats.primal_feas: norm of current feasibility;
    • stats.iter: current iteration counter;
    • stats.objective: current objective function value;
    • stats.solution: current iterate;
    • stats.multipliers: current Lagrange multipliers estimate;
    • stats.multipliers_L and stats.multipliers_U: current Lagrange multipliers estimate for the lower and upper bounds respectively;
    • stats.status: current status of the algorithm. Should be :unknown unless the algorithm has attained a stopping criterion. Changing this to anything will stop the algorithm, but you should use :user to properly indicate the intention.
    • stats.elapsed_time: elapsed time in seconds.

Examples

julia> using FletcherPenaltySolver, ADNLPModels
 julia> nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0]);
 julia> stats = fps_solve(nlp)
-"Execution stats: first-order stationary"
source
FletcherPenaltySolver.ghjvprod_nln! — Method
ghjvprod_nln!(nlp::FletcherPenaltyNLP, x, y, v, Hv; obj_weight = one(S)) where {S}

Redefine the NLPModel function ghjvprod to account for Lagrange multiplier of size < ncon.

source
FletcherPenaltySolver.hess_nln — Method
hess_nln(nlp::FletcherPenaltyNLP, x, y; obj_weight = one(S)) where {S}

Redefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.

source
FletcherPenaltySolver.hess_nln_coord! — Method
hess_nln_nln!(nlp::FletcherPenaltyNLP, x, y, vals; obj_weight = one(S)) where {S}

Redefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.

source
FletcherPenaltySolver.hprod_nln! — Method
hprod_nln!(nlp::FletcherPenaltyNLP, x, y, v, Hv; obj_weight = one(S)) where {S}

Redefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.

source
FletcherPenaltySolver.solve_two_extras — Function
invJtJJv, invJtJSsv = solve_two_extras(nlp, x, rhs1, rhs2)

The IterativeSolver variant solve successively a regularized least square, see solve_least_square, and a regularized minres. It returns only a warning if the method failed.

The LDLtSolver variant use successively a regularized cgls and a regularized minres.

source
FletcherPenaltySolver.solve_two_least_squares — Function
p1, q1, p2, q2 = solve_two_least_squares(nlp, x, rhs1, rhs2)

Solve successively two least square regularized by √nlp.δ:

\[ min || ∇c' q - rhs || + δ || q ||^2\]

rhs1 and rhs2 are both of size nlp.meta.nvar.

The IterativeSolver variant uses two calls to a Krylov.jl method, see solve_least_square. Note that nlp.Aop is not re-evaluated in this case. It returns only a warning if the method failed.

The LDLtSolver variant use an LDLt factorization to solve the large system.

source
FletcherPenaltySolver.solve_two_mixed — Function
p1, q1, p2, q2 = solve_two_mixed(nlp, x, rhs1, rhs2)

Solve successively a least square regularized by √nlp.δ:

\[ min || ∇c' q - rhs || + δ || q ||^2\]

and a least-norm problem.

rhs1 is of size nlp.meta.nvar, and rhs2 is of size nlp.meta.ncon.

The IterativeSolver variant uses two calls to a Krylov.jl method, see solve_least_square and solve_least_norm. It returns only a warning if the method failed.

The LDLtSolver variant use an LDLt factorization to solve the large system.

source
+"Execution stats: first-order stationary"source
FletcherPenaltySolver.ghjvprod_nln! — Method
ghjvprod_nln!(nlp::FletcherPenaltyNLP, x, y, v, Hv; obj_weight = one(S)) where {S}

Redefine the NLPModel function ghjvprod to account for Lagrange multiplier of size < ncon.

source
FletcherPenaltySolver.go_log — Method
go_log(stp, sub_stp, fx, ncx, mess::String, verbose)

Logging shortcut.

source
FletcherPenaltySolver.hess_nln — Method
hess_nln(nlp::FletcherPenaltyNLP, x, y; obj_weight = one(S)) where {S}

Redefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.

source
FletcherPenaltySolver.hess_nln_coord! — Method
hess_nln_nln!(nlp::FletcherPenaltyNLP, x, y, vals; obj_weight = one(S)) where {S}

Redefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.

source
FletcherPenaltySolver.hprod_nln! — Method
hprod_nln!(nlp::FletcherPenaltyNLP, x, y, v, Hv; obj_weight = one(S)) where {S}

Redefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.

source
FletcherPenaltySolver.linear_system2 — Method
p1, q1, p2, q2 = linear_system2(nlp, x)

Call to solve_two_mixed(nlp, x, nlp.gx, nlp.cx), see solve_two_mixed.

source
FletcherPenaltySolver.random_restoration! — Method
random_restoration!(meta, stp, sub_stp)

Add a random perturbation to the current iterate.

source
FletcherPenaltySolver.restoration_feasibility! — Method
restoration_feasibility!(feasibility_solver, meta, stp, sub_stp, feas_tol, ncx, verbose)

Try to find a feasible point, see feasibility_step.

source
FletcherPenaltySolver.solve_least_norm — Method
solve_least_norm(qdsolver::IterativeSolver, A, b, λ)

Solve least squares problem with regularization δ.

source
FletcherPenaltySolver.solve_least_square — Method
solve_least_square(qdsolver::IterativeSolver, A, b, λ)

Solve least squares problem with regularization λ.

source
FletcherPenaltySolver.solve_two_extras — Function
invJtJJv, invJtJSsv = solve_two_extras(nlp, x, rhs1, rhs2)

The IterativeSolver variant solve successively a regularized least square, see solve_least_square, and a regularized minres. It returns only a warning if the method failed.

The LDLtSolver variant use successively a regularized cgls and a regularized minres.

source
FletcherPenaltySolver.solve_two_least_squares — Function
p1, q1, p2, q2 = solve_two_least_squares(nlp, x, rhs1, rhs2)

Solve successively two least square regularized by √nlp.δ:

\[ min || ∇c' q - rhs || + δ || q ||^2\]

rhs1 and rhs2 are both of size nlp.meta.nvar.

The IterativeSolver variant uses two calls to a Krylov.jl method, see solve_least_square. Note that nlp.Aop is not re-evaluated in this case. It returns only a warning if the method failed.

The LDLtSolver variant use an LDLt factorization to solve the large system.

source
FletcherPenaltySolver.solve_two_mixed — Function
p1, q1, p2, q2 = solve_two_mixed(nlp, x, rhs1, rhs2)

Solve successively a least square regularized by √nlp.δ:

\[ min || ∇c' q - rhs || + δ || q ||^2\]

and a least-norm problem.

rhs1 is of size nlp.meta.nvar, and rhs2 is of size nlp.meta.ncon.

The IterativeSolver variant uses two calls to a Krylov.jl method, see solve_least_square and solve_least_norm. It returns only a warning if the method failed.

The LDLtSolver variant use an LDLt factorization to solve the large system.

source
FletcherPenaltySolver.update_parameters! — Method
update_parameters!(meta, sub_stp, feas)

Update σ. If the current iterate also update ρ.

source
FletcherPenaltySolver.update_parameters_unbdd! — Method
update_parameters_unbdd!(meta, sub_stp, feas)

Start or update δ, then call update_parameters!(meta, sub_stp, feas)

source
diff --git a/previews/PR134/search/index.html b/previews/PR134/search/index.html index cef7a73..c85970c 100644 --- a/previews/PR134/search/index.html +++ b/previews/PR134/search/index.html @@ -1,2 +1,2 @@ -Search · FletcherPenaltySolver

Loading search...

    +Search · FletcherPenaltySolver

    Loading search...

      diff --git a/previews/PR134/search_index.js b/previews/PR134/search_index.js index e43a726..06c9e26 100644 --- a/previews/PR134/search_index.js +++ b/previews/PR134/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"fine-tuneFPS/#Advanced-usage-of-FletcherPenaltySolver.jl","page":"Fine-tune FPS","title":"Advanced usage of FletcherPenaltySolver.jl","text":"","category":"section"},{"location":"fine-tuneFPS/#Contents","page":"Fine-tune FPS","title":"Contents","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Pages = [\"fine-tuneFPS.md\"]","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The main function exported by this package is the function fps_solve whose basic usage has been illustrated previously. It is also possible to fine-tune the parameters used in the implementation in two different ways.","category":"page"},{"location":"fine-tuneFPS/#Examples","page":"Fine-tune FPS","title":"Examples","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"FletcherPenaltySolver.jl exports the function fps_solve:","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":" fps_solve(nlp::AbstractNLPModel, x0::AbstractVector{T} = nlp.meta.x0; verbose::Int = 0, subsolver_verbose::Int = 0, lagrange_bound = 1 / sqrt(eps(T)), kwargs...)\n fps_solve(stp::NLPStopping; verbose::Int = 0, subsolver_verbose::Int = 0, lagrange_bound = 1 / sqrt(eps()), kwargs...)\n fps_solve(stp::NLPStopping, fpssolver::FPSSSolver{T, QDS, US}; verbose::Int = 0, subsolver_verbose::Int = 0, lagrange_bound::T = 1 / sqrt(eps(T)))","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"It is, therefore, possible to either call fps_solve(nlp, x, kwargs...) and the keywords arguments are passed to both NLPStopping and/or FPSSSolver constructor or build an instance of NLPStopping and/or FPSSSolver directly.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"using ADNLPModels, FletcherPenaltySolver\n\nnlp = ADNLPModel(\n x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, \n [-1.2; 1.0],\n x->[x[1] * x[2] - 1], \n [0.0], [0.0],\n name = \"Rosenbrock with x₁x₂=1\"\n)\nstats = fps_solve(nlp)","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The alternative using NLPStopping, see Stopping.jl, allow to reuse the same memory if one would re-solve a problem of the same dimension","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"using ADNLPModels, FletcherPenaltySolver, Stopping\nstp = NLPStopping(nlp)\nstats = fps_solve(stp)\nstp.current_state.x .= rand(2)\nstats = fps_solve(stp)","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The FPSSSolver, see FPSSSolver, contains all the metadata and additional pre-allocated memory used by the algorithm.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"using SolverCore\nstp = NLPStopping(nlp)\ndata = FPSSSolver(stp)\nstats = GenericExecutionStats(nlp)\nstats = SolverCore.solve!(data, stp, stats)","category":"page"},{"location":"fine-tuneFPS/#List-of-possible-options","page":"Fine-tune FPS","title":"List of possible options","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Find below a list of the main options of fps_solve.","category":"page"},{"location":"fine-tuneFPS/#Tolerances-on-the-problem","page":"Fine-tune FPS","title":"Tolerances on the problem","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"We use Stopping.jl to control the algorithmic flow, we refer to Stopping.jl and https://solverstoppingjulia.github.io for tutorials and documentation. By default, we use the function Fletcher_penalty_optimality_check as optimality check, and the default tol_check is rtol [1 + c(x₀); 1 + ∇f(x₀)] with rtol = 1e-7.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Additional parameters used in stopping the algorithm are defined in the following table.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Parameters Type Default Description\nlagrange_bound Real 1 / sqrt(eps(T)) bounds on estimated Lagrange multipliers.\nsubsolver_max_iter Real 20000 maximum iteration for the subproblem solver.","category":"page"},{"location":"fine-tuneFPS/#Algorithmic-parameters","page":"Fine-tune FPS","title":"Algorithmic parameters","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The metadata is defined in a AlgoData structure at the initialization of FPSSolver.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Parameters Type Default Description\nσ_0 Real 1e3 Initialize subproblem's parameter σ\nσ_max Real 1 / √eps(T) Maximum value for subproblem's parameter σ\nσ_update Real T(2) Update subproblem's parameter σ\nρ_0 Real T(2) Initialize subproblem's parameter ρ\nρ_max Real 1 / √eps(T) Maximum value for subproblem's parameter ρ\nρ_update Real T(2) Update subproblem's parameter ρ\nδ_0 Real √eps(T) Initialize subproblem's parameter δ\nδ_max Real 1 / √eps(T) Maximum value for subproblem's parameter δ\nδ_update Real T(10) Update subproblem's parameter δ\nη_1 Real zero(T) Initialize subproblem's parameter η\nη_update Real one(T) Update subproblem's parameter η\nyM Real typemax(T) bound on the Lagrange multipliers\nΔ Real T(0.95) expected decrease in feasibility between two iterations\nsubproblem_solver Function KNITRO.has_knitro() ? NLPModelsKnitro.knitro : ipopt solver used for the subproblem, see also JSOSolvers.jl\nsubpb_unbounded_threshold Real 1 / √eps(T) below the opposite of this value, the subproblem is unbounded\natol_sub Function atol -> atol absolute tolerance for the subproblem in function of atol\nrtol_sub Function rtol -> rtol relative tolerance for the subproblem in function of rtol\nhessian_approx either Val(1) or Val(2) Val(2) it selects the hessian approximation\nconvex_subproblem Bool false true if the subproblem is convex. Useful to set the convex option in knitro.\nqds_solver Symbol :ldlt Initialize the QDSolver to solve quasi-definite systems, either :ldlt or :iterative.","category":"page"},{"location":"fine-tuneFPS/#Feasibility-step","page":"Fine-tune FPS","title":"Feasibility step","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The metadata for the feasibility procedure is defined in a GNSolver structure at the initialization of FPSSolver.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Parameters Type Default Description\nη₁ Real 1e-3 Feasibility step: decrease the trust-region radius when Ared/Pred < η₁.\nη₂ Real 0.66 Feasibility step: increase the trust-region radius when Ared/Pred > η₂.\nσ₁ Real 0.25 Feasibility step: decrease coefficient of the trust-region radius.\nσ₂ Real 2.0 Feasibility step: increase coefficient of the trust-region radius.\nΔ₀ Real 1.0 Feasibility step: initial radius.\nfeas_expected_decrease Real 0.95 Feasibility step: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feasexpecteddecrease.\nbad_steps_lim Integer 3 Feasibility step: consecutive bad steps before using a second order step.\nTR_compute_step KrylovSolver LsmrSolver Compute the direction in feasibility step.\naggressive_step KrylovSolver CgSolver Compute the (aggressive) direction in feasibility step.","category":"page"},{"location":"reference/#Reference","page":"Reference","title":"Reference","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/#Contents","page":"Reference","title":"Contents","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Pages = [\"reference.md\"]","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/#Index","page":"Reference","title":"Index","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Pages = [\"reference.md\"]","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Modules = [FletcherPenaltySolver]","category":"page"},{"location":"reference/#FletcherPenaltySolver.AlgoData","page":"Reference","title":"FletcherPenaltySolver.AlgoData","text":"AlgoData(; kwargs...) \nAlgoData(T::DataType; kwargs...)\n\nStructure containing all the parameters used in the fps_solve call. T is the datatype used in the algorithm, by default it is Float64. Returns a AlgoData structure.\n\nArguments\n\nThe keyword arguments may include:\n\nσ_0::Real = T(1e3): Initialize subproblem's parameter σ;\nσ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter σ;\nσ_update::Real = T(2): Update subproblem's parameter σ;\nρ_0::Real = one(T): Initialize subproblem's parameter ρ;\nρ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter ρ;\nρ_update::Real = T(2): Update subproblem's parameter ρ;\nδ_0::Real = √eps(T): Initialize subproblem's parameter δ;\nδ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter δ;\nδ_update::Real = T(10): Update subproblem's parameter δ;\nη_1::Real = zero(T): Initialize subproblem's parameter η;\nη_update::Real = one(T): Update subproblem's parameter η;\nyM::Real = typemax(T): bound on the Lagrange multipliers;\nΔ::Real = T(0.95): expected decrease in feasibility between two iterations;\nsubproblem_solver::Function = ipopt: solver used for the subproblem;\nsubpb_unbounded_threshold::Real = 1 / √eps(T): below the opposite of this value, the subproblem is unbounded;\nsubsolver_max_iter::Int = 20000: maximum of iteration for the subproblem solver;\natol_sub::Function = atol -> atol: absolute tolerance for the subproblem in function of atol;\nrtol_sub::Function = rtol -> rtol: relative tolerance for the subproblem in function of rtol;\nhessian_approx = Val(2): either Val(1) or Val(2), it selects the hessian approximation;\nconvex_subproblem::Bool = false: true if the subproblem is convex. Useful to set the convex option in knitro;\nlagrange_bound::T = 1 / sqrt(eps(T)): upper-bound on the Lagrange multiplier.\n\nFor more details, we refer to the package documentation fine-tuneFPS.md. \n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.DirectSolver","page":"Reference","title":"FletcherPenaltySolver.DirectSolver","text":"DirectSolver(nlp::AbstractNLPModel, ::T) <: QDSolver\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.FPSSSolver","page":"Reference","title":"FletcherPenaltySolver.FPSSSolver","text":"FPSSSolver(nlp::AbstractNLPModel [, x0 = nlp.meta.x0]; kwargs...)\nFPSSSolver(stp::NLPStopping; kwargs...)\n\nStructure regrouping all the structure used during the fps_solve call. It returns a FPSSSolver structure.\n\nArguments\n\nThe keyword arguments may include:\n\nstp::NLPStopping: Stopping structure for this algorithm workflow;\nmeta::AlgoData{T}: see AlgoData;\nworkspace: allocated space for the solver itself;\nqdsolver: solver structure for the linear algebra part, contains allocation for this part. By default a LDLtSolver, but an alternative is IterativeSolver ;\nsubproblem_solver::AbstractOptimizationSolver: by default a subproblem_solver_correspondence[Symbol(meta.subproblem_solver)];\nsub_stats::GenericExecutionStats: stats structure for the result of subproblem_solver;\nfeasibility_solver: by default a GNSolver, see GNSolver;\nmodel::FletcherPenaltyNLP: subproblem;\nsub_stp::NLPStopping: Stopping structure for the subproblem.\n\nNote:\n\nsubproblem_solver is accessible from the subproblem_solver_correspondence::Dict.\nthe qdsolver is accessible from the dictionary qdsolver_correspondence.\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.FletcherPenaltyNLP","page":"Reference","title":"FletcherPenaltySolver.FletcherPenaltyNLP","text":"FletcherPenaltyNLP(nlp, σ, hessian_approx, [x0 = nlp.meta.x0]; qds = LDLtSolver(nlp, S(0)))\nFletcherPenaltyNLP(nlp, σ, ρ, δ, hessian_approx, [x0 = nlp.meta.x0]; qds = LDLtSolver(nlp, S(0)))\nFletcherPenaltyNLP(nlp; σ_0 = 1, ρ_0 = 0, δ_0 = 0, hessian_approx = Val(2), x0 = nlp.meta.x0, qds = LDLtSolver(nlp, S(0)))\n\nWe consider here the implementation of Fletcher's exact penalty method for the minimization problem:\n\n minₓ f(x) st c(x) = ℓ l x u\n\nusing Fletcher penalty function:\n\n minₓ f(x) - (c(x) - ℓ)^T ys(x) + 05 ρ c(x) - ℓ²₂ st l x u\n\nwhere\n\n ys(x) arg min_y 05 A(x)y - g(x)²₂ + σ (c(x) - ℓ)^T y + 05 δ ²₂\n\nArguments\n\nnlp::AbstractNLPModel: the model solved, see NLPModels.jl;\nx::AbstractVector: Initial guess. If x is not specified, then nlp.meta.x0 is used;\nσ, ρ, δ parameters of the subproblem;\nhessian_approx either Val(1) or Val(2) for the hessian approximation.\nqds: solver structure for the linear algebra computations, see LDLtSolver or IterativeSolver.\n\nNotes:\n\nEvaluation of the obj, grad, objgrad functions evaluate functions from the orginial nlp. These values are stored in fx, cx, gx.\nThe value of the penalty vector ys is also stored.\nThe hessian's structure is dense.\n\nExamples\n\njulia> using FletcherPenaltySolver, ADNLPModels\njulia> nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0])\njulia> fp_sos = FletcherPenaltyNLP(nlp)\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.GNSolver","page":"Reference","title":"FletcherPenaltySolver.GNSolver","text":"GNSolver(x, y; kwargs...)\n\nStructure containing all the parameters used in the feasibility step. x is an intial guess, and y is an initial guess for the Lagrange multiplier. Returns a GNSolver structure.\n\nArguments\n\nThe keyword arguments may include:\n\nη₁::T=T(1e-3): Feasibility step: decrease the trust-region radius when Ared/Pred < η₁.\nη₂::T=T(0.66): Feasibility step: increase the trust-region radius when Ared/Pred > η₂.\nσ₁::T=T(0.25): Feasibility step: decrease coefficient of the trust-region radius.\nσ₂::T=T(2.0): Feasibility step: increase coefficient of the trust-region radius.\nΔ₀::T=one(T): Feasibility step: initial radius.\nbad_steps_lim::Integer=3: Feasibility step: consecutive bad steps before using a second order step.\nfeas_expected_decrease::T=T(0.95): Feasibility step: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feas_expected_decrease.\nTR_compute_step = LsmrSolver(length(y0), length(x0), S): Compute the direction in feasibility step.\naggressive_step = CgSolver(length(x0), length(x0), S): Compute the direction in feasibility step in agressive mode.\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.IterativeSolver","page":"Reference","title":"FletcherPenaltySolver.IterativeSolver","text":"IterativeSolver(nlp::AbstractNLPModel, ::T) <: QDSolver\n\nIt uses Krylov.jl methods to solve least-squares and least-norm problems.\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.LDLtSolver","page":"Reference","title":"FletcherPenaltySolver.LDLtSolver","text":"LDLtSolver(nlp::AbstractNLPModel, ::T) <: QDSolver\n\nIt uses LDLFactorization.jl methods to solve least-squares and least-norm problems.\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.LUSolver","page":"Reference","title":"FletcherPenaltySolver.LUSolver","text":"LUSolver(nlp::AbstractNLPModel, ::T) <: QDSolver\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.QDSolver","page":"Reference","title":"FletcherPenaltySolver.QDSolver","text":"QDSolver\n\nAbstract structure handling parameters for the system\n\n In A \n A -nlpδ*Im \n\nthat we are solving twice.\n\nIts implementation should define:\n\nsolve_two_extras\nsolve_two_least_squares\nsolve_two_mixed\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.Fletcher_penalty_optimality_check-Union{Tuple{S}, Tuple{T}, Tuple{NLPModels.AbstractNLPModel{T, S}, Stopping.NLPAtX}} where {T, S}","page":"Reference","title":"FletcherPenaltySolver.Fletcher_penalty_optimality_check","text":"Fletcher_penalty_optimality_check(pb::AbstractNLPModel, state::NLPAtX)\n\nOptimality function used by default in the algorithm. An alternative is to use the function KKT from Stopping.jl.\n\nThe function returns a vector of length ncon + nvar containing:\n\n|c(x) - lcon| / |x|₂\nres / |λ|₂ ; x - max(min(x - res, uvar), lvar)) if it has bounds\n\nThe fields x, cx and res need to be filled. If state.lambda is nothing then we take |λ|₂=1.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.TR_lsmr-Union{Tuple{T}, Tuple{Any, AbstractVector{T}, Union{AbstractMatrix{T}, LinearOperators.LinearOperator{T}}, AbstractFloat, T, AbstractFloat, AbstractVector{T}}} where T","page":"Reference","title":"FletcherPenaltySolver.TR_lsmr","text":"TR_lsmr(solver, cz, Jz, ctol, Δ, normcz, Jd)\n\nCompute a direction d such that\n\nbeginaligned\nmin_d quad c + Jz d \ntextst quad d leq Delta\nendaligned\n\nusing lsmr method from Krylov.jl.\n\nOutput\n\nd: solution\nJd: product of the solution with J.\ninfeasible: true if the problem is infeasible.\nsolved: true if the problem has been successfully solved.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver._compute_ys_gs!-Union{Tuple{T}, Tuple{FletcherPenaltyNLP, AbstractVector{T}}} where T","page":"Reference","title":"FletcherPenaltySolver._compute_ys_gs!","text":"gs, ys, v, w = _compute_ys_gs!(nlp, x)\n\nCompute the Lagrange multipliers and the gradient of the Lagrangian function in-place.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.cons_norhs!-Tuple{FletcherPenaltyNLP, Any, Any}","page":"Reference","title":"FletcherPenaltySolver.cons_norhs!","text":"cons_norhs!(nlp::FletcherPenaltyNLP, x, cx)\n\nRedefine the NLPModel function cons! to account for non-zero right-hand side in the equation constraint. It returns cons(nlp, x) - nlp.meta.lcon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.feasibility_step-Union{Tuple{T}, Tuple{FletcherPenaltySolver.GNSolver, NLPModels.AbstractNLPModel, AbstractVector{T}, AbstractVector{T}, T, Union{AbstractMatrix{T}, LinearOperators.LinearOperator{T}}, T, AbstractFloat, Any}} where T","page":"Reference","title":"FletcherPenaltySolver.feasibility_step","text":"feasibility_step(feasibility_solver, nlp, x, cx, normcx, Jx, ρ, ctol; kwargs...)\n\nApproximately solves min ‖c(x) - l‖, where l is nlp.meta.lcon, using a trust-region Levenberg-Marquardt method.\n\nArguments\n\nη₁::AbstractFloat = feasibility_solver.feas_η₁: decrease the trust-region radius when Ared/Pred < η₁.\nη₂::AbstractFloat = feasibility_solver.feas_η₂: increase the trust-region radius when Ared/Pred > η₂.\nσ₁::AbstractFloat = feasibility_solver.feas_σ₁: decrease coefficient of the trust-region radius.\nσ₂::AbstractFloat = feasibility_solver.feas_σ₂:increase coefficient of the trust-region radius.\nΔ₀::T = feasibility_solver.feas_Δ₀: initial trust-region radius.\nbad_steps_lim::Integer = feasibility_solver.bad_steps_lim: consecutive bad steps before using a second order step.\nexpected_decrease::T = feasibility_solver.feas_expected_decrease: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feas_expected_decrease.\nmax_eval::Int = 1_000: maximum evaluations.\nmax_time::AbstractFloat = 60.0: maximum time.\nmax_feas_iter::Int = typemax(Int64): maximum number of iterations.\n\nOutput\n\nz, cz, normcz, Jz: the new iterate, and updated evaluations.\nstatus: Computation status. Possible outcomes are: :success, max_eval, max_time, max_iter, unknown_tired, :infeasible, :unknown.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.fps_solve-Union{Tuple{NLPModels.AbstractNLPModel{T, V}}, Tuple{V}, Tuple{T}, Tuple{NLPModels.AbstractNLPModel{T, V}, V}} where {T, V}","page":"Reference","title":"FletcherPenaltySolver.fps_solve","text":"fps_solve(nlp::AbstractNLPModel, x0::AbstractVector{T} = nlp.meta.x0; subsolver_verbose::Int = 0, kwargs...)\n\nCompute a local minimum of a bound and equality-constrained optimization problem using Fletcher's penalty function and the implementation described in\n\nEstrin, R., Friedlander, M. P., Orban, D., & Saunders, M. A. (2020).\nImplementing a smooth exact penalty function for equality-constrained nonlinear optimization.\nSIAM Journal on Scientific Computing, 42(3), A1809-A1835.\nhttps://doi.org/10.1137/19M1238265\n\nFor advanced usage, the principal call to the solver uses a NLPStopping, see Stopping.jl.\n\nfps_solve(stp::NLPStopping, fpssolver::FPSSSolver{T, QDS, US}; subsolver_verbose::Int = 0)\nfps_solve(stp::NLPStopping; subsolver_verbose::Int = 0, kwargs...)\n\nArguments\n\nnlp::AbstractNLPModel: the model solved, see NLPModels.jl;\nx: Initial guess. If x is not specified, then nlp.meta.x0 is used.\n\nKeyword arguments\n\nfpssolver: see FPSSSolver;\nverbose::Int = 0: if > 0, display iteration information of the solver;\nsubsolver_verbose::Int = 0: if > 0, display iteration information of the subsolver;\n\nAll the information regarding stopping criteria can be set in the NLPStopping object. Additional kwargs are given to the NLPStopping. By default, the optimality condition used to declare optimality is Fletcher_penalty_optimality_check.\n\nOutput\n\nThe returned value is a GenericExecutionStats, see SolverCore.jl.\n\nIf one define a Stopping before calling fps_solve, it is possible to access all the information computed by the algorithm.\n\nNotes\n\nIf the problem has inequalities, we use slack variables to get only equalities and bounds via NLPModelsModifiers.jl.\nstp.current_state.res contains the gradient of Fletcher's penalty function.\nsubproblem_solver must take an NLPStopping as input, see StoppingInterface.jl.\n\nCallback\n\nThe callback is called at each iteration. The expected signature of the callback is callback(nlp, solver, stats), and its output is ignored. Changing any of the input arguments will affect the subsequent iterations. In particular, setting stats.status = :user will stop the algorithm. All relevant information should be available in nlp and solver. Notably, you can access, and modify, the following:\n\nsolver: see FPSSSolver\nstats: structure holding the output of the algorithm (GenericExecutionStats), which contains, among other things:\nstats.dual_feas: norm of current gradient of the Lagrangian;\nstats.primal_feas: norm of current feasibility;\nstats.iter: current iteration counter;\nstats.objective: current objective function value;\nstats.solution: current iterate;\nstats.multipliers: current Lagrange multipliers estimate;\nstats.multipliers_L and stats.multipliers_U: current Lagrange multipliers estimate for the lower and upper bounds respectively;\nstats.status: current status of the algorithm. Should be :unknown unless the algorithm has attained a stopping criterion. Changing this to anything will stop the algorithm, but you should use :user to properly indicate the intention.\nstats.elapsed_time: elapsed time in seconds.\n\nExamples\n\njulia> using FletcherPenaltySolver, ADNLPModels\njulia> nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0]);\njulia> stats = fps_solve(nlp)\n\"Execution stats: first-order stationary\"\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.ghjvprod_nln!-Union{Tuple{S}, Tuple{FletcherPenaltyNLP, AbstractVector{S}, Any, Any, Any}} where S","page":"Reference","title":"FletcherPenaltySolver.ghjvprod_nln!","text":"ghjvprod_nln!(nlp::FletcherPenaltyNLP, x, y, v, Hv; obj_weight = one(S)) where {S}\n\nRedefine the NLPModel function ghjvprod to account for Lagrange multiplier of size < ncon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.go_log-Tuple{Any, Any, Any, Any, String, Any}","page":"Reference","title":"FletcherPenaltySolver.go_log","text":"go_log(stp, sub_stp, fx, ncx, mess::String, verbose)\n\nLogging shortcut.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.hess_nln-Union{Tuple{S}, Tuple{FletcherPenaltyNLP, AbstractVector{S}, Any}} where S","page":"Reference","title":"FletcherPenaltySolver.hess_nln","text":"hess_nln(nlp::FletcherPenaltyNLP, x, y; obj_weight = one(S)) where {S}\n\nRedefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.hess_nln_coord!-Union{Tuple{S}, Tuple{FletcherPenaltyNLP, AbstractVector{S}, Any, Any}} where S","page":"Reference","title":"FletcherPenaltySolver.hess_nln_coord!","text":"hess_nln_nln!(nlp::FletcherPenaltyNLP, x, y, vals; obj_weight = one(S)) where {S}\n\nRedefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.hprod_nln!-Union{Tuple{S}, Tuple{FletcherPenaltyNLP, AbstractVector{S}, Any, Any, Any}} where S","page":"Reference","title":"FletcherPenaltySolver.hprod_nln!","text":"hprod_nln!(nlp::FletcherPenaltyNLP, x, y, v, Hv; obj_weight = one(S)) where {S}\n\nRedefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.linear_system2-Union{Tuple{T}, Tuple{FletcherPenaltyNLP, AbstractVector{T}}} where T","page":"Reference","title":"FletcherPenaltySolver.linear_system2","text":"p1, q1, p2, q2 = linear_system2(nlp, x)\n\nCall to solve_two_mixed(nlp, x, nlp.gx, nlp.cx), see solve_two_mixed.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.random_restoration!-Tuple{Any, Any, Any}","page":"Reference","title":"FletcherPenaltySolver.random_restoration!","text":"random_restoration!(meta, stp, sub_stp)\n\nAdd a random perturbation to the current iterate.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.restoration_feasibility!-NTuple{7, Any}","page":"Reference","title":"FletcherPenaltySolver.restoration_feasibility!","text":"restoration_feasibility!(feasibility_solver, meta, stp, sub_stp, feas_tol, ncx, verbose)\n\nTry to find a feasible point, see feasibility_step.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.solve_least_norm-Union{Tuple{It}, Tuple{SS3}, Tuple{SS2}, Tuple{SS1}, Tuple{S}, Tuple{T}, Tuple{FletcherPenaltySolver.IterativeSolver{T, S, SS1, SS2, SS3, It}, Any, Any, Any}} where {T, S, SS1, SS2, SS3, It}","page":"Reference","title":"FletcherPenaltySolver.solve_least_norm","text":"solve_least_norm(qdsolver::IterativeSolver, A, b, λ)\n\nSolve least squares problem with regularization δ.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.solve_least_square-Union{Tuple{It}, Tuple{SS3}, Tuple{SS2}, Tuple{SS1}, Tuple{S}, Tuple{T}, Tuple{FletcherPenaltySolver.IterativeSolver{T, S, SS1, SS2, SS3, It}, Any, Any, Any}} where {T, S, SS1, SS2, SS3, It}","page":"Reference","title":"FletcherPenaltySolver.solve_least_square","text":"solve_least_square(qdsolver::IterativeSolver, A, b, λ)\n\nSolve least squares problem with regularization λ.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.solve_two_extras","page":"Reference","title":"FletcherPenaltySolver.solve_two_extras","text":"invJtJJv, invJtJSsv = solve_two_extras(nlp, x, rhs1, rhs2)\n\nThe IterativeSolver variant solve successively a regularized least square, see solve_least_square, and a regularized minres. It returns only a warning if the method failed.\n\nThe LDLtSolver variant use successively a regularized cgls and a regularized minres.\n\n\n\n\n\n","category":"function"},{"location":"reference/#FletcherPenaltySolver.solve_two_least_squares","page":"Reference","title":"FletcherPenaltySolver.solve_two_least_squares","text":"p1, q1, p2, q2 = solve_two_least_squares(nlp, x, rhs1, rhs2)\n\nSolve successively two least square regularized by √nlp.δ:\n\n min c q - rhs + δ q ^2\n\nrhs1 and rhs2 are both of size nlp.meta.nvar.\n\nThe IterativeSolver variant uses two calls to a Krylov.jl method, see solve_least_square. Note that nlp.Aop is not re-evaluated in this case. It returns only a warning if the method failed.\n\nThe LDLtSolver variant use an LDLt factorization to solve the large system.\n\n\n\n\n\n","category":"function"},{"location":"reference/#FletcherPenaltySolver.solve_two_mixed","page":"Reference","title":"FletcherPenaltySolver.solve_two_mixed","text":"p1, q1, p2, q2 = solve_two_mixed(nlp, x, rhs1, rhs2)\n\nSolve successively a least square regularized by √nlp.δ:\n\n min c q - rhs + δ q ^2\n\nand a least-norm problem.\n\nrhs1 is of size nlp.meta.nvar, and rhs2 is of size nlp.meta.ncon.\n\nThe IterativeSolver variant uses two calls to a Krylov.jl method, see solve_least_square and solve_least_norm. It returns only a warning if the method failed.\n\nThe LDLtSolver variant use an LDLt factorization to solve the large system.\n\n\n\n\n\n","category":"function"},{"location":"reference/#FletcherPenaltySolver.update_parameters!-Tuple{Any, Any, Any}","page":"Reference","title":"FletcherPenaltySolver.update_parameters!","text":"update_parameters!(meta, sub_stp, feas)\n\nUpdate σ. If the current iterate also update ρ.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.update_parameters_unbdd!-Tuple{Any, Any, Any}","page":"Reference","title":"FletcherPenaltySolver.update_parameters_unbdd!","text":"update_parameters_unbdd!(meta, sub_stp, feas)\n\nStart or update δ, then call update_parameters!(meta, sub_stp, feas)\n\n\n\n\n\n","category":"method"},{"location":"#FletcherPenaltySolver.jl-Fletcher's-Penalty-Method","page":"Introduction","title":"FletcherPenaltySolver.jl - Fletcher's Penalty Method","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"FPS is a solver for equality-constrained nonlinear problems, i.e., optimization problems of the form","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":" min_x f(x) quad textst quad c(x) = 0","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"It uses other JuliaSmoothOptimizers packages for development. In particular, NLPModels.jl is used for defining the problem, and SolverCore.jl for the output. If a general inequality-constrained problem is given to the solver, it solves the problem reformulated as a SlackModel from NLPModelsModifiers.jl.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"We refer to juliasmoothoptimizers.github.io for tutorials on the NLPModel API. This framework allows the usage of models from Ampl (using AmplNLReader.jl), CUTEst (using CUTEst.jl), JuMP (using NLPModelsJuMP.jl), PDE-constrained optimization problems (using PDENLPModels.jl) and models defined with automatic differentiation (using ADNLPModels.jl).","category":"page"},{"location":"#Algorithm","page":"Introduction","title":"Algorithm","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"The function fps_solver solves a nonlinear optimization problem by iteratively solving the bound-constrained optimization problem using Fletcher penalty function:","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":" beginaligned\n min_x f(x) - c(x)^T lambda_delta(x) + fracrho2c(x)^2_2 \n mboxwhere lambda_delta(x) in textargmin_y frac12 nabla c(x)^T y - nabla f(x) ^2_2 + sigma c(x)^T y + fracdelta2y^2\n endaligned","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"For equality-constrained problems, the method iteratively solves an unconstrained problem. For bound and equality-constrained problems, the subproblems are bound-constrained problems. Any solver compatible with Stopping.jl can be used. By default, we use ipopt from NLPModelsIpopt.jl to solve the subproblem, but other solvers can be used such as knitro from NLPModelsKnitro.jl or any solvers from JSOSolvers.jl. The Stopping version of these solvers is available in StoppingInterface.jl.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"It uses LDLFactorizations.jl by default to evaluate the derivatives of the penalized subproblem, but one can also use a matrix-free version with Krylov.jl.","category":"page"},{"location":"#References","page":"Introduction","title":"References","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"Estrin, R., Friedlander, M. P., Orban, D., & Saunders, M. A. (2020). Implementing a smooth exact penalty function for equality-constrained nonlinear optimization. SIAM Journal on Scientific Computing, 42(3), A1809-A1835. 10.1137/19M1238265","category":"page"},{"location":"#How-to-Cite","page":"Introduction","title":"How to Cite","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"If you use FletcherPenaltySolver.jl in your work, please cite using the format given in CITATION.cff.","category":"page"},{"location":"#Installation","page":"Introduction","title":"Installation","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"pkg> add https://github.com/JuliaSmoothOptimizers/FletcherPenaltySolver.jl","category":"page"},{"location":"#Example","page":"Introduction","title":"Example","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"We consider in this example the minization of the Rosenbrock function.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":" min_x 100 (x₂ - x₁²)² + (x₁ - 1)²","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"The problem is modeled using ADNLPModels.jl with [-1.2; 1.0] as default initial point, and then solved using fps_solve.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"using FletcherPenaltySolver, ADNLPModels\n\n# Rosenbrock\nnlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0])\nstats = fps_solve(nlp)","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"We consider in this example the minization of the Rosenbrock function over an inequality constraint.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":" min_x 100 (x₂ - x₁²)² + (x₁ - 1)² quad textst quad 0 x₁x₂ 1","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"The problem is modeled using ADNLPModels.jl with [-1.2; 1.0] as default initial point, and then solved using fps_solve.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"using FletcherPenaltySolver, ADNLPModels\n\nnlp = ADNLPModel(\n x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2,\n [-1.2; 1.0],\n x->[x[1] * x[2]],\n [0.0],\n [1.0],\n)\nstats = fps_solve(nlp)","category":"page"},{"location":"#Bug-reports-and-discussions","page":"Introduction","title":"Bug reports and discussions","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers, so questions about any of our packages are welcome.","category":"page"},{"location":"tutorial/#Tutorial","page":"Tutorial","title":"Tutorial","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Pages = [\"tutorial.md\"]","category":"page"},{"location":"tutorial/#FletcherPenaltySolver-Tutorial","page":"Tutorial","title":"FletcherPenaltySolver Tutorial","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"In this tutorial, we explore on small instances various possibilities offered by fps_solve defined in the package FletcherPenaltySolver.","category":"page"},{"location":"tutorial/#Type-stable-algorithm","page":"Tutorial","title":"Type stable algorithm","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The algorithm is implemented in pure Julia, so if one also chooses an unconstrained solver in pure Julia, we can Julia's type stability to solve optimization problems in a precision different than Float64. In the following example, we use tron from JSOSolvers.jl on a simple example in Float32.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using ADNLPModels, FletcherPenaltySolver, JSOSolvers\nT = Float32\nnlp = ADNLPModel(x -> (1 - x[1])^2, T[-1.2; 1.0], x -> [10 * (x[2] - x[1]^2)], T[0.0], T[0.0])\nstats = fps_solve(nlp, hessian_approx = Val(2), subproblem_solver = tron, rtol = T(1e-4), verbose = 1)\n(stats.dual_feas, stats.primal_feas, stats.status, typeof(stats.solution))","category":"page"},{"location":"tutorial/#A-factorization-free-solver","page":"Tutorial","title":"A factorization-free solver","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The main advantage of fps_solver is the possibility to use Hessian and Jacobian-vector products only, whenever one uses a subproblem solver with the same property. So, it is not necessary to compute and store explicitly those matrices. In the following example, we choose a problem with equality constraints from OptimizationProblems.jl.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using ADNLPModels, FletcherPenaltySolver, JSOSolvers, OptimizationProblems\nnlp = OptimizationProblems.ADNLPProblems.hs28()\nstats = fps_solve(nlp, subproblem_solver = tron, qds_solver = :iterative)\n(stats.dual_feas, stats.primal_feas, stats.status, stats.elapsed_time)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Exploring nlp's counter, we can see that no Hessian or Jacobian matrix has been evaluated.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"nlp.counters","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"We can compare this result with ipopt that uses the Jacobian and Hessian matrices.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using NLPModels, NLPModelsIpopt\nreset!(nlp);\nstats = fps_solve(nlp, subproblem_solver = ipopt, qds_solver = :iterative)\n(stats.dual_feas, stats.primal_feas, stats.status, stats.elapsed_time)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"nlp.counters","category":"page"},{"location":"tutorial/#Stopping-solver","page":"Tutorial","title":"Stopping-solver","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using ADNLPModels, FletcherPenaltySolver, Stopping\nf(x) = (x[1] - 1)^2 + 4 * (x[2] - x[1]^2)^2\nc(x) = [x[1]^2 + x[2]^2 - 2]\nT = Float64\nx0 = T[-1.2; 1.0]\nℓ, u = zeros(T, 2), 2 * ones(T, 2)\nnlp = ADNLPModel(f, x0, ℓ, u, c, zeros(1), zeros(1))\n\nstp = NLPStopping(nlp)\nstats = fps_solve(stp)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"It is then possible to explore the various quantities computed by the algorithm. For instance, recompute the gradient of the Lagrangian.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"state = stp.current_state\nstate.gx + state.Jx' * state.lambda","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Another possibility is to reuse the Stopping for another solve.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"new_x0 = 4 * ones(2)\nreinit!(stp, rstate = true, x = new_x0)\nStopping.reset!(stp.pb)\nstats = fps_solve(stp)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"We refer to Stopping.jl and https://solverstoppingjulia.github.io for tutorials and documentation.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"EditURL = \"../assets/example.jl\"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"(Image: Binder)","category":"page"},{"location":"example/#Solve-Large-Scale-Problem-with-FletcherPenaltySolver.jl","page":"Large-scale example","title":"Solve Large-Scale Problem with FletcherPenaltySolver.jl","text":"","category":"section"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"In this tutorial we use fps_solve to solve a large-scale optimization problem resulting from the discretization of a PDE-constrained optimization problem and compare the solve with Ipopt.","category":"page"},{"location":"example/#Problem-Statement","page":"Large-scale example","title":"Problem Statement","text":"","category":"section"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Let Ω = (-1,1)², we solve the following distributed Poisson control problem with Dirichlet boundary:","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":" leftlbrace\n beginaligned\n min_y in H^1_0 u in H^1 quad frac12 int_Omega y(x) - y_d(x)^2dx + fracalpha2 int_Omega u^2dx \n textst -Delta y = h + u quad x in Omega \n y = 0 quad x in partial Omega\n endaligned\n right","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"where yd(x) = -x₁² and α = 1e-2. The force term is h(x₁, x₂) = - sin(ω x₁)sin(ω x₂) with ω = π - 1/8.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"We refer to Gridap.jl for more details on modeling PDEs and PDENLPModels.jl for PDE-constrained optimization problems.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"using Gridap, PDENLPModels","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"WARNING: Method definition testargs(Gridap.Arrays.PosNegReindex{A, B} where B where A, Integer) in module Arrays at /home/runner/.julia/packages/Gridap/EZQEK/src/Arrays/PosNegReindex.jl:10 overwritten in module PDENLPModels at /home/runner/.julia/packages/PDENLPModels/pW0Iv/src/PDENLPModels.jl:16.\nERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.\nWARNING: method definition for _compute_hess_structure at /home/runner/.julia/packages/PDENLPModels/pW0Iv/src/hessian_struct_nnzh_functions.jl:70 declares type variable T but does not use it.\nWARNING: method definition for _compute_hess_structure at /home/runner/.julia/packages/PDENLPModels/pW0Iv/src/hessian_struct_nnzh_functions.jl:74 declares type variable T but does not use it.\n","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Definition of the domain and discretization","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"n = 20\ndomain = (-1, 1, -1, 1)\npartition = (n, n)\nmodel = CartesianDiscreteModel(domain, partition)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"CartesianDiscreteModel()","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Definition of the FE-spaces","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"reffe = ReferenceFE(lagrangian, Float64, 2)\nXpde = TestFESpace(model, reffe; conformity = :H1, dirichlet_tags = \"boundary\")\ny0(x) = 0.0\nYpde = TrialFESpace(Xpde, y0)\n\nreffe_con = ReferenceFE(lagrangian, Float64, 1)\nXcon = TestFESpace(model, reffe_con; conformity = :H1)\nYcon = TrialFESpace(Xcon)\nY = MultiFieldFESpace([Ypde, Ycon])","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"MultiFieldFESpace()","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Integration machinery","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"trian = Triangulation(model)\ndegree = 1\ndΩ = Measure(trian, degree)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Measure()","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Objective function","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"yd(x) = -x[1]^2\nα = 1e-2\nfunction f(y, u)\n ∫(0.5 * (yd - y) * (yd - y) + 0.5 * α * u * u) * dΩ\nend","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"f (generic function with 1 method)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Definition of the constraint operator","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"ω = π - 1 / 8\nh(x) = -sin(ω * x[1]) * sin(ω * x[2])\nfunction res(y, u, v)\n ∫(∇(v) ⊙ ∇(y) - v * u - v * h) * dΩ\nend\nop = FEOperator(res, Y, Xpde)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"FEOperatorFromWeakForm()","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Definition of the initial guess","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"npde = Gridap.FESpaces.num_free_dofs(Ypde)\nncon = Gridap.FESpaces.num_free_dofs(Ycon)\nx0 = zeros(npde + ncon);","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Overall, we built a GridapPDENLPModel, which implements the NLPModels.jl API.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"nlp = GridapPDENLPModel(x0, f, trian, Ypde, Ycon, Xpde, Xcon, op, name = \"Control elastic membrane\")\n\n(nlp.meta.nvar, nlp.meta.ncon)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"(1962, 1521)","category":"page"},{"location":"example/#Find-a-Feasible-Point","page":"Large-scale example","title":"Find a Feasible Point","text":"","category":"section"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Before solving the previously defined model, we will first improve our initial guess. We use FeasibilityResidual from NLPModelsModifiers.jl to convert the NLPModel as an NLSModel. Then, using trunk, a solver for least-squares problems implemented in JSOSolvers.jl, we find An improved guess which is close to being feasible for our large-scale problem. By default, a JSO-compliant solver such as trunk (the same applies to fps_solve) uses by default nlp.meta.x0 as an initial guess.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"using JSOSolvers, NLPModelsModifiers\n\nnls = FeasibilityResidual(nlp)\nstats_trunk = trunk(nls)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"\"Execution stats: first-order stationary\"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"We check the solution from the stats returned by trunk:","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"norm(cons(nlp, stats_trunk.solution))","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"1.6058259477686155e-5","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"We will use the solution found to initialize our solvers.","category":"page"},{"location":"example/#Solve-the-Problem","page":"Large-scale example","title":"Solve the Problem","text":"","category":"section"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Finally, we are ready to solve the PDE-constrained optimization problem with a targeted tolerance of 1e-5. In the following, we will use both Ipopt and DCI on our problem.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"using NLPModelsIpopt\n\nstats_ipopt = ipopt(nlp, x0 = stats_trunk.solution, tol = 1e-5, print_level = 0)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"\"Execution stats: first-order stationary\"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"The problem was successfully solved, and we can extract the function evaluations from the stats.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"nlp.counters","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":" Counters:\n obj: ██████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 9 grad: ███████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 10 cons: ████████████⋅⋅⋅⋅⋅⋅⋅⋅ 18 \n cons_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 cons_nln: ████████████⋅⋅⋅⋅⋅⋅⋅⋅ 18 jcon: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jgrad: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jac: ███████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 10 jac_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jac_nln: ███████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 10 jprod: ███████████████⋅⋅⋅⋅⋅ 22 jprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jprod_nln: ███████████████⋅⋅⋅⋅⋅ 22 jtprod: ████████████████████ 30 jtprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jtprod_nln: ████████████████████ 30 hess: ██████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 8 hprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jhess: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jhprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Reinitialize the counters before re-solving.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"reset!(nlp);","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"NullLogger avoids printing iteration information.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"using FletcherPenaltySolver, Logging\n\nstats_fps_solve = with_logger(NullLogger()) do\n fps_solve(nlp, stats_trunk.solution, atol = 1e-5, rtol = 1e-5)\nend","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"\"Execution stats: first-order stationary\"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"The problem was successfully solved, and we can extract the function evaluations from the stats.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"nlp.counters","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":" Counters:\n obj: ██████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 3 grad: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 cons: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 \n cons_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 cons_nln: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 jcon: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jgrad: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jac: ████████████⋅⋅⋅⋅⋅⋅⋅⋅ 6 jac_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jac_nln: ████████████⋅⋅⋅⋅⋅⋅⋅⋅ 6 jprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jprod_nln: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jtprod: ██████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5 jtprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jtprod_nln: ██████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5 hess: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 hprod: ████████████████████ 10 \n jhess: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jhprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"We now compare the two solvers with respect to the time spent,","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"stats_ipopt.elapsed_time, stats_fps_solve.elapsed_time","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"(17.899, 23.588104963302612)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"and also check objective value, feasibility and dual feasibility of ipopt and fps_solve.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"(stats_ipopt.objective, stats_ipopt.primal_feas, stats_ipopt.dual_feas),\n(stats_fps_solve.objective, stats_fps_solve.primal_feas, stats_fps_solve.dual_feas)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"((0.005425026428348349, 2.2204460492503135e-18, 4.2724309076258577e-7), (0.005425025968573663, 2.2204460492503135e-18, 2.968652346957424e-7))","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Overall FletcherPenaltySolver is doing great for solving large-scale optimization problems!","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"This page was generated using Literate.jl.","category":"page"}] +[{"location":"fine-tuneFPS/#Advanced-usage-of-FletcherPenaltySolver.jl","page":"Fine-tune FPS","title":"Advanced usage of FletcherPenaltySolver.jl","text":"","category":"section"},{"location":"fine-tuneFPS/#Contents","page":"Fine-tune FPS","title":"Contents","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Pages = [\"fine-tuneFPS.md\"]","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The main function exported by this package is the function fps_solve whose basic usage has been illustrated previously. It is also possible to fine-tune the parameters used in the implementation in two different ways.","category":"page"},{"location":"fine-tuneFPS/#Examples","page":"Fine-tune FPS","title":"Examples","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"FletcherPenaltySolver.jl exports the function fps_solve:","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":" fps_solve(nlp::AbstractNLPModel, x0::AbstractVector{T} = nlp.meta.x0; verbose::Int = 0, subsolver_verbose::Int = 0, lagrange_bound = 1 / sqrt(eps(T)), kwargs...)\n fps_solve(stp::NLPStopping; verbose::Int = 0, subsolver_verbose::Int = 0, lagrange_bound = 1 / sqrt(eps()), kwargs...)\n fps_solve(stp::NLPStopping, fpssolver::FPSSSolver{T, QDS, US}; verbose::Int = 0, subsolver_verbose::Int = 0, lagrange_bound::T = 1 / sqrt(eps(T)))","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"It is, therefore, possible to either call fps_solve(nlp, x, kwargs...) and the keywords arguments are passed to both NLPStopping and/or FPSSSolver constructor or build an instance of NLPStopping and/or FPSSSolver directly.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"using ADNLPModels, FletcherPenaltySolver\n\nnlp = ADNLPModel(\n x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, \n [-1.2; 1.0],\n x->[x[1] * x[2] - 1], \n [0.0], [0.0],\n name = \"Rosenbrock with x₁x₂=1\"\n)\nstats = fps_solve(nlp)","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The alternative using NLPStopping, see Stopping.jl, allow to reuse the same memory if one would re-solve a problem of the same dimension","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"using ADNLPModels, FletcherPenaltySolver, Stopping\nstp = NLPStopping(nlp)\nstats = fps_solve(stp)\nstp.current_state.x .= rand(2)\nstats = fps_solve(stp)","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The FPSSSolver, see FPSSSolver, contains all the metadata and additional pre-allocated memory used by the algorithm.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"using SolverCore\nstp = NLPStopping(nlp)\ndata = FPSSSolver(stp)\nstats = GenericExecutionStats(nlp)\nstats = SolverCore.solve!(data, stp, stats)","category":"page"},{"location":"fine-tuneFPS/#List-of-possible-options","page":"Fine-tune FPS","title":"List of possible options","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Find below a list of the main options of fps_solve.","category":"page"},{"location":"fine-tuneFPS/#Tolerances-on-the-problem","page":"Fine-tune FPS","title":"Tolerances on the problem","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"We use Stopping.jl to control the algorithmic flow, we refer to Stopping.jl and https://solverstoppingjulia.github.io for tutorials and documentation. By default, we use the function Fletcher_penalty_optimality_check as optimality check, and the default tol_check is rtol [1 + c(x₀); 1 + ∇f(x₀)] with rtol = 1e-7.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Additional parameters used in stopping the algorithm are defined in the following table.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Parameters Type Default Description\nlagrange_bound Real 1 / sqrt(eps(T)) bounds on estimated Lagrange multipliers.\nsubsolver_max_iter Real 20000 maximum iteration for the subproblem solver.","category":"page"},{"location":"fine-tuneFPS/#Algorithmic-parameters","page":"Fine-tune FPS","title":"Algorithmic parameters","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The metadata is defined in a AlgoData structure at the initialization of FPSSolver.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Parameters Type Default Description\nσ_0 Real 1e3 Initialize subproblem's parameter σ\nσ_max Real 1 / √eps(T) Maximum value for subproblem's parameter σ\nσ_update Real T(2) Update subproblem's parameter σ\nρ_0 Real T(2) Initialize subproblem's parameter ρ\nρ_max Real 1 / √eps(T) Maximum value for subproblem's parameter ρ\nρ_update Real T(2) Update subproblem's parameter ρ\nδ_0 Real √eps(T) Initialize subproblem's parameter δ\nδ_max Real 1 / √eps(T) Maximum value for subproblem's parameter δ\nδ_update Real T(10) Update subproblem's parameter δ\nη_1 Real zero(T) Initialize subproblem's parameter η\nη_update Real one(T) Update subproblem's parameter η\nyM Real typemax(T) bound on the Lagrange multipliers\nΔ Real T(0.95) expected decrease in feasibility between two iterations\nsubproblem_solver Function KNITRO.has_knitro() ? NLPModelsKnitro.knitro : ipopt solver used for the subproblem, see also JSOSolvers.jl\nsubpb_unbounded_threshold Real 1 / √eps(T) below the opposite of this value, the subproblem is unbounded\natol_sub Function atol -> atol absolute tolerance for the subproblem in function of atol\nrtol_sub Function rtol -> rtol relative tolerance for the subproblem in function of rtol\nhessian_approx either Val(1) or Val(2) Val(2) it selects the hessian approximation\nconvex_subproblem Bool false true if the subproblem is convex. Useful to set the convex option in knitro.\nqds_solver Symbol :ldlt Initialize the QDSolver to solve quasi-definite systems, either :ldlt or :iterative.","category":"page"},{"location":"fine-tuneFPS/#Feasibility-step","page":"Fine-tune FPS","title":"Feasibility step","text":"","category":"section"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"The metadata for the feasibility procedure is defined in a GNSolver structure at the initialization of FPSSolver.","category":"page"},{"location":"fine-tuneFPS/","page":"Fine-tune FPS","title":"Fine-tune FPS","text":"Parameters Type Default Description\nη₁ Real 1e-3 Feasibility step: decrease the trust-region radius when Ared/Pred < η₁.\nη₂ Real 0.66 Feasibility step: increase the trust-region radius when Ared/Pred > η₂.\nσ₁ Real 0.25 Feasibility step: decrease coefficient of the trust-region radius.\nσ₂ Real 2.0 Feasibility step: increase coefficient of the trust-region radius.\nΔ₀ Real 1.0 Feasibility step: initial radius.\nfeas_expected_decrease Real 0.95 Feasibility step: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feasexpecteddecrease.\nbad_steps_lim Integer 3 Feasibility step: consecutive bad steps before using a second order step.\nTR_compute_step KrylovSolver LsmrSolver Compute the direction in feasibility step.\naggressive_step KrylovSolver CgSolver Compute the (aggressive) direction in feasibility step.","category":"page"},{"location":"reference/#Reference","page":"Reference","title":"Reference","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/#Contents","page":"Reference","title":"Contents","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Pages = [\"reference.md\"]","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/#Index","page":"Reference","title":"Index","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Pages = [\"reference.md\"]","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"​","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Modules = [FletcherPenaltySolver]","category":"page"},{"location":"reference/#FletcherPenaltySolver.AlgoData","page":"Reference","title":"FletcherPenaltySolver.AlgoData","text":"AlgoData(; kwargs...) \nAlgoData(T::DataType; kwargs...)\n\nStructure containing all the parameters used in the fps_solve call. T is the datatype used in the algorithm, by default it is Float64. Returns a AlgoData structure.\n\nArguments\n\nThe keyword arguments may include:\n\nσ_0::Real = T(1e3): Initialize subproblem's parameter σ;\nσ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter σ;\nσ_update::Real = T(2): Update subproblem's parameter σ;\nρ_0::Real = one(T): Initialize subproblem's parameter ρ;\nρ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter ρ;\nρ_update::Real = T(2): Update subproblem's parameter ρ;\nδ_0::Real = √eps(T): Initialize subproblem's parameter δ;\nδ_max::Real = 1 / √eps(T): Maximum value for subproblem's parameter δ;\nδ_update::Real = T(10): Update subproblem's parameter δ;\nη_1::Real = zero(T): Initialize subproblem's parameter η;\nη_update::Real = one(T): Update subproblem's parameter η;\nyM::Real = typemax(T): bound on the Lagrange multipliers;\nΔ::Real = T(0.95): expected decrease in feasibility between two iterations;\nsubproblem_solver::Function = ipopt: solver used for the subproblem;\nsubpb_unbounded_threshold::Real = 1 / √eps(T): below the opposite of this value, the subproblem is unbounded;\nsubsolver_max_iter::Int = 20000: maximum of iteration for the subproblem solver;\natol_sub::Function = atol -> atol: absolute tolerance for the subproblem in function of atol;\nrtol_sub::Function = rtol -> rtol: relative tolerance for the subproblem in function of rtol;\nhessian_approx = Val(2): either Val(1) or Val(2), it selects the hessian approximation;\nconvex_subproblem::Bool = false: true if the subproblem is convex. Useful to set the convex option in knitro;\nlagrange_bound::T = 1 / sqrt(eps(T)): upper-bound on the Lagrange multiplier.\n\nFor more details, we refer to the package documentation fine-tuneFPS.md. \n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.DirectSolver","page":"Reference","title":"FletcherPenaltySolver.DirectSolver","text":"DirectSolver(nlp::AbstractNLPModel, ::T) <: QDSolver\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.FPSSSolver","page":"Reference","title":"FletcherPenaltySolver.FPSSSolver","text":"FPSSSolver(nlp::AbstractNLPModel [, x0 = nlp.meta.x0]; kwargs...)\nFPSSSolver(stp::NLPStopping; kwargs...)\n\nStructure regrouping all the structure used during the fps_solve call. It returns a FPSSSolver structure.\n\nArguments\n\nThe keyword arguments may include:\n\nstp::NLPStopping: Stopping structure for this algorithm workflow;\nmeta::AlgoData{T}: see AlgoData;\nworkspace: allocated space for the solver itself;\nqdsolver: solver structure for the linear algebra part, contains allocation for this part. By default a LDLtSolver, but an alternative is IterativeSolver ;\nsubproblem_solver::AbstractOptimizationSolver: by default a subproblem_solver_correspondence[Symbol(meta.subproblem_solver)];\nsub_stats::GenericExecutionStats: stats structure for the result of subproblem_solver;\nfeasibility_solver: by default a GNSolver, see GNSolver;\nmodel::FletcherPenaltyNLP: subproblem;\nsub_stp::NLPStopping: Stopping structure for the subproblem.\n\nNote:\n\nsubproblem_solver is accessible from the subproblem_solver_correspondence::Dict.\nthe qdsolver is accessible from the dictionary qdsolver_correspondence.\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.FletcherPenaltyNLP","page":"Reference","title":"FletcherPenaltySolver.FletcherPenaltyNLP","text":"FletcherPenaltyNLP(nlp, σ, hessian_approx, [x0 = nlp.meta.x0]; qds = LDLtSolver(nlp, S(0)))\nFletcherPenaltyNLP(nlp, σ, ρ, δ, hessian_approx, [x0 = nlp.meta.x0]; qds = LDLtSolver(nlp, S(0)))\nFletcherPenaltyNLP(nlp; σ_0 = 1, ρ_0 = 0, δ_0 = 0, hessian_approx = Val(2), x0 = nlp.meta.x0, qds = LDLtSolver(nlp, S(0)))\n\nWe consider here the implementation of Fletcher's exact penalty method for the minimization problem:\n\n minₓ f(x) st c(x) = ℓ l x u\n\nusing Fletcher penalty function:\n\n minₓ f(x) - (c(x) - ℓ)^T ys(x) + 05 ρ c(x) - ℓ²₂ st l x u\n\nwhere\n\n ys(x) arg min_y 05 A(x)y - g(x)²₂ + σ (c(x) - ℓ)^T y + 05 δ ²₂\n\nArguments\n\nnlp::AbstractNLPModel: the model solved, see NLPModels.jl;\nx::AbstractVector: Initial guess. If x is not specified, then nlp.meta.x0 is used;\nσ, ρ, δ parameters of the subproblem;\nhessian_approx either Val(1) or Val(2) for the hessian approximation.\nqds: solver structure for the linear algebra computations, see LDLtSolver or IterativeSolver.\n\nNotes:\n\nEvaluation of the obj, grad, objgrad functions evaluate functions from the orginial nlp. These values are stored in fx, cx, gx.\nThe value of the penalty vector ys is also stored.\nThe hessian's structure is dense.\n\nExamples\n\njulia> using FletcherPenaltySolver, ADNLPModels\njulia> nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0])\njulia> fp_sos = FletcherPenaltyNLP(nlp)\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.GNSolver","page":"Reference","title":"FletcherPenaltySolver.GNSolver","text":"GNSolver(x, y; kwargs...)\n\nStructure containing all the parameters used in the feasibility step. x is an intial guess, and y is an initial guess for the Lagrange multiplier. Returns a GNSolver structure.\n\nArguments\n\nThe keyword arguments may include:\n\nη₁::T=T(1e-3): Feasibility step: decrease the trust-region radius when Ared/Pred < η₁.\nη₂::T=T(0.66): Feasibility step: increase the trust-region radius when Ared/Pred > η₂.\nσ₁::T=T(0.25): Feasibility step: decrease coefficient of the trust-region radius.\nσ₂::T=T(2.0): Feasibility step: increase coefficient of the trust-region radius.\nΔ₀::T=one(T): Feasibility step: initial radius.\nbad_steps_lim::Integer=3: Feasibility step: consecutive bad steps before using a second order step.\nfeas_expected_decrease::T=T(0.95): Feasibility step: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feas_expected_decrease.\nTR_compute_step = LsmrSolver(length(y0), length(x0), S): Compute the direction in feasibility step.\naggressive_step = CgSolver(length(x0), length(x0), S): Compute the direction in feasibility step in agressive mode.\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.IterativeSolver","page":"Reference","title":"FletcherPenaltySolver.IterativeSolver","text":"IterativeSolver(nlp::AbstractNLPModel, ::T) <: QDSolver\n\nIt uses Krylov.jl methods to solve least-squares and least-norm problems.\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.LDLtSolver","page":"Reference","title":"FletcherPenaltySolver.LDLtSolver","text":"LDLtSolver(nlp::AbstractNLPModel, ::T) <: QDSolver\n\nIt uses LDLFactorization.jl methods to solve least-squares and least-norm problems.\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.LUSolver","page":"Reference","title":"FletcherPenaltySolver.LUSolver","text":"LUSolver(nlp::AbstractNLPModel, ::T) <: QDSolver\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.QDSolver","page":"Reference","title":"FletcherPenaltySolver.QDSolver","text":"QDSolver\n\nAbstract structure handling parameters for the system\n\n In A \n A -nlpδ*Im \n\nthat we are solving twice.\n\nIts implementation should define:\n\nsolve_two_extras\nsolve_two_least_squares\nsolve_two_mixed\n\n\n\n\n\n","category":"type"},{"location":"reference/#FletcherPenaltySolver.Fletcher_penalty_optimality_check-Union{Tuple{S}, Tuple{T}, Tuple{NLPModels.AbstractNLPModel{T, S}, Stopping.NLPAtX}} where {T, S}","page":"Reference","title":"FletcherPenaltySolver.Fletcher_penalty_optimality_check","text":"Fletcher_penalty_optimality_check(pb::AbstractNLPModel, state::NLPAtX)\n\nOptimality function used by default in the algorithm. An alternative is to use the function KKT from Stopping.jl.\n\nThe function returns a vector of length ncon + nvar containing:\n\n|c(x) - lcon| / |x|₂\nres / |λ|₂ ; x - max(min(x - res, uvar), lvar)) if it has bounds\n\nThe fields x, cx and res need to be filled. If state.lambda is nothing then we take |λ|₂=1.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.TR_lsmr-Union{Tuple{T}, Tuple{Any, AbstractVector{T}, Union{AbstractMatrix{T}, LinearOperators.LinearOperator{T}}, AbstractFloat, T, AbstractFloat, AbstractVector{T}}} where T","page":"Reference","title":"FletcherPenaltySolver.TR_lsmr","text":"TR_lsmr(solver, cz, Jz, ctol, Δ, normcz, Jd)\n\nCompute a direction d such that\n\nbeginaligned\nmin_d quad c + Jz d \ntextst quad d leq Delta\nendaligned\n\nusing lsmr method from Krylov.jl.\n\nOutput\n\nd: solution\nJd: product of the solution with J.\ninfeasible: true if the problem is infeasible.\nsolved: true if the problem has been successfully solved.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver._compute_ys_gs!-Union{Tuple{T}, Tuple{FletcherPenaltyNLP, AbstractVector{T}}} where T","page":"Reference","title":"FletcherPenaltySolver._compute_ys_gs!","text":"gs, ys, v, w = _compute_ys_gs!(nlp, x)\n\nCompute the Lagrange multipliers and the gradient of the Lagrangian function in-place.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.cons_norhs!-Tuple{FletcherPenaltyNLP, Any, Any}","page":"Reference","title":"FletcherPenaltySolver.cons_norhs!","text":"cons_norhs!(nlp::FletcherPenaltyNLP, x, cx)\n\nRedefine the NLPModel function cons! to account for non-zero right-hand side in the equation constraint. It returns cons(nlp, x) - nlp.meta.lcon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.feasibility_step-Union{Tuple{T}, Tuple{FletcherPenaltySolver.GNSolver, NLPModels.AbstractNLPModel, AbstractVector{T}, AbstractVector{T}, T, Union{AbstractMatrix{T}, LinearOperators.LinearOperator{T}}, T, AbstractFloat, Any}} where T","page":"Reference","title":"FletcherPenaltySolver.feasibility_step","text":"feasibility_step(feasibility_solver, nlp, x, cx, normcx, Jx, ρ, ctol; kwargs...)\n\nApproximately solves min ‖c(x) - l‖, where l is nlp.meta.lcon, using a trust-region Levenberg-Marquardt method.\n\nArguments\n\nη₁::AbstractFloat = feasibility_solver.feas_η₁: decrease the trust-region radius when Ared/Pred < η₁.\nη₂::AbstractFloat = feasibility_solver.feas_η₂: increase the trust-region radius when Ared/Pred > η₂.\nσ₁::AbstractFloat = feasibility_solver.feas_σ₁: decrease coefficient of the trust-region radius.\nσ₂::AbstractFloat = feasibility_solver.feas_σ₂:increase coefficient of the trust-region radius.\nΔ₀::T = feasibility_solver.feas_Δ₀: initial trust-region radius.\nbad_steps_lim::Integer = feasibility_solver.bad_steps_lim: consecutive bad steps before using a second order step.\nexpected_decrease::T = feasibility_solver.feas_expected_decrease: bad steps are when ‖c(z)‖ / ‖c(x)‖ >feas_expected_decrease.\nmax_eval::Int = 1_000: maximum evaluations.\nmax_time::AbstractFloat = 60.0: maximum time.\nmax_feas_iter::Int = typemax(Int64): maximum number of iterations.\n\nOutput\n\nz, cz, normcz, Jz: the new iterate, and updated evaluations.\nstatus: Computation status. Possible outcomes are: :success, max_eval, max_time, max_iter, unknown_tired, :infeasible, :unknown.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.fps_solve-Union{Tuple{NLPModels.AbstractNLPModel{T, V}}, Tuple{V}, Tuple{T}, Tuple{NLPModels.AbstractNLPModel{T, V}, V}} where {T, V}","page":"Reference","title":"FletcherPenaltySolver.fps_solve","text":"fps_solve(nlp::AbstractNLPModel, x0::AbstractVector{T} = nlp.meta.x0; subsolver_verbose::Int = 0, kwargs...)\n\nCompute a local minimum of a bound and equality-constrained optimization problem using Fletcher's penalty function and the implementation described in\n\nEstrin, R., Friedlander, M. P., Orban, D., & Saunders, M. A. (2020).\nImplementing a smooth exact penalty function for equality-constrained nonlinear optimization.\nSIAM Journal on Scientific Computing, 42(3), A1809-A1835.\nhttps://doi.org/10.1137/19M1238265\n\nFor advanced usage, the principal call to the solver uses a NLPStopping, see Stopping.jl.\n\nfps_solve(stp::NLPStopping, fpssolver::FPSSSolver{T, QDS, US}; subsolver_verbose::Int = 0)\nfps_solve(stp::NLPStopping; subsolver_verbose::Int = 0, kwargs...)\n\nArguments\n\nnlp::AbstractNLPModel: the model solved, see NLPModels.jl;\nx: Initial guess. If x is not specified, then nlp.meta.x0 is used.\n\nKeyword arguments\n\nfpssolver: see FPSSSolver;\nverbose::Int = 0: if > 0, display iteration information of the solver;\nsubsolver_verbose::Int = 0: if > 0, display iteration information of the subsolver;\n\nAll the information regarding stopping criteria can be set in the NLPStopping object. Additional kwargs are given to the NLPStopping. By default, the optimality condition used to declare optimality is Fletcher_penalty_optimality_check.\n\nOutput\n\nThe returned value is a GenericExecutionStats, see SolverCore.jl.\n\nIf one define a Stopping before calling fps_solve, it is possible to access all the information computed by the algorithm.\n\nNotes\n\nIf the problem has inequalities, we use slack variables to get only equalities and bounds via NLPModelsModifiers.jl.\nstp.current_state.res contains the gradient of Fletcher's penalty function.\nsubproblem_solver must take an NLPStopping as input, see StoppingInterface.jl.\n\nCallback\n\nThe callback is called at each iteration. The expected signature of the callback is callback(nlp, solver, stats), and its output is ignored. Changing any of the input arguments will affect the subsequent iterations. In particular, setting stats.status = :user will stop the algorithm. All relevant information should be available in nlp and solver. Notably, you can access, and modify, the following:\n\nsolver: see FPSSSolver\nstats: structure holding the output of the algorithm (GenericExecutionStats), which contains, among other things:\nstats.dual_feas: norm of current gradient of the Lagrangian;\nstats.primal_feas: norm of current feasibility;\nstats.iter: current iteration counter;\nstats.objective: current objective function value;\nstats.solution: current iterate;\nstats.multipliers: current Lagrange multipliers estimate;\nstats.multipliers_L and stats.multipliers_U: current Lagrange multipliers estimate for the lower and upper bounds respectively;\nstats.status: current status of the algorithm. Should be :unknown unless the algorithm has attained a stopping criterion. Changing this to anything will stop the algorithm, but you should use :user to properly indicate the intention.\nstats.elapsed_time: elapsed time in seconds.\n\nExamples\n\njulia> using FletcherPenaltySolver, ADNLPModels\njulia> nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0]);\njulia> stats = fps_solve(nlp)\n\"Execution stats: first-order stationary\"\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.ghjvprod_nln!-Union{Tuple{S}, Tuple{FletcherPenaltyNLP, AbstractVector{S}, Any, Any, Any}} where S","page":"Reference","title":"FletcherPenaltySolver.ghjvprod_nln!","text":"ghjvprod_nln!(nlp::FletcherPenaltyNLP, x, y, v, Hv; obj_weight = one(S)) where {S}\n\nRedefine the NLPModel function ghjvprod to account for Lagrange multiplier of size < ncon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.go_log-Tuple{Any, Any, Any, Any, String, Any}","page":"Reference","title":"FletcherPenaltySolver.go_log","text":"go_log(stp, sub_stp, fx, ncx, mess::String, verbose)\n\nLogging shortcut.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.hess_nln-Union{Tuple{S}, Tuple{FletcherPenaltyNLP, AbstractVector{S}, Any}} where S","page":"Reference","title":"FletcherPenaltySolver.hess_nln","text":"hess_nln(nlp::FletcherPenaltyNLP, x, y; obj_weight = one(S)) where {S}\n\nRedefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.hess_nln_coord!-Union{Tuple{S}, Tuple{FletcherPenaltyNLP, AbstractVector{S}, Any, Any}} where S","page":"Reference","title":"FletcherPenaltySolver.hess_nln_coord!","text":"hess_nln_nln!(nlp::FletcherPenaltyNLP, x, y, vals; obj_weight = one(S)) where {S}\n\nRedefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.hprod_nln!-Union{Tuple{S}, Tuple{FletcherPenaltyNLP, AbstractVector{S}, Any, Any, Any}} where S","page":"Reference","title":"FletcherPenaltySolver.hprod_nln!","text":"hprod_nln!(nlp::FletcherPenaltyNLP, x, y, v, Hv; obj_weight = one(S)) where {S}\n\nRedefine the NLPModel function hprod to account for Lagrange multiplier of size < ncon.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.linear_system2-Union{Tuple{T}, Tuple{FletcherPenaltyNLP, AbstractVector{T}}} where T","page":"Reference","title":"FletcherPenaltySolver.linear_system2","text":"p1, q1, p2, q2 = linear_system2(nlp, x)\n\nCall to solve_two_mixed(nlp, x, nlp.gx, nlp.cx), see solve_two_mixed.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.random_restoration!-Tuple{Any, Any, Any}","page":"Reference","title":"FletcherPenaltySolver.random_restoration!","text":"random_restoration!(meta, stp, sub_stp)\n\nAdd a random perturbation to the current iterate.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.restoration_feasibility!-NTuple{7, Any}","page":"Reference","title":"FletcherPenaltySolver.restoration_feasibility!","text":"restoration_feasibility!(feasibility_solver, meta, stp, sub_stp, feas_tol, ncx, verbose)\n\nTry to find a feasible point, see feasibility_step.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.solve_least_norm-Union{Tuple{It}, Tuple{SS3}, Tuple{SS2}, Tuple{SS1}, Tuple{S}, Tuple{T}, Tuple{FletcherPenaltySolver.IterativeSolver{T, S, SS1, SS2, SS3, It}, Any, Any, Any}} where {T, S, SS1, SS2, SS3, It}","page":"Reference","title":"FletcherPenaltySolver.solve_least_norm","text":"solve_least_norm(qdsolver::IterativeSolver, A, b, λ)\n\nSolve least squares problem with regularization δ.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.solve_least_square-Union{Tuple{It}, Tuple{SS3}, Tuple{SS2}, Tuple{SS1}, Tuple{S}, Tuple{T}, Tuple{FletcherPenaltySolver.IterativeSolver{T, S, SS1, SS2, SS3, It}, Any, Any, Any}} where {T, S, SS1, SS2, SS3, It}","page":"Reference","title":"FletcherPenaltySolver.solve_least_square","text":"solve_least_square(qdsolver::IterativeSolver, A, b, λ)\n\nSolve least squares problem with regularization λ.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.solve_two_extras","page":"Reference","title":"FletcherPenaltySolver.solve_two_extras","text":"invJtJJv, invJtJSsv = solve_two_extras(nlp, x, rhs1, rhs2)\n\nThe IterativeSolver variant solve successively a regularized least square, see solve_least_square, and a regularized minres. It returns only a warning if the method failed.\n\nThe LDLtSolver variant use successively a regularized cgls and a regularized minres.\n\n\n\n\n\n","category":"function"},{"location":"reference/#FletcherPenaltySolver.solve_two_least_squares","page":"Reference","title":"FletcherPenaltySolver.solve_two_least_squares","text":"p1, q1, p2, q2 = solve_two_least_squares(nlp, x, rhs1, rhs2)\n\nSolve successively two least square regularized by √nlp.δ:\n\n min c q - rhs + δ q ^2\n\nrhs1 and rhs2 are both of size nlp.meta.nvar.\n\nThe IterativeSolver variant uses two calls to a Krylov.jl method, see solve_least_square. Note that nlp.Aop is not re-evaluated in this case. It returns only a warning if the method failed.\n\nThe LDLtSolver variant use an LDLt factorization to solve the large system.\n\n\n\n\n\n","category":"function"},{"location":"reference/#FletcherPenaltySolver.solve_two_mixed","page":"Reference","title":"FletcherPenaltySolver.solve_two_mixed","text":"p1, q1, p2, q2 = solve_two_mixed(nlp, x, rhs1, rhs2)\n\nSolve successively a least square regularized by √nlp.δ:\n\n min c q - rhs + δ q ^2\n\nand a least-norm problem.\n\nrhs1 is of size nlp.meta.nvar, and rhs2 is of size nlp.meta.ncon.\n\nThe IterativeSolver variant uses two calls to a Krylov.jl method, see solve_least_square and solve_least_norm. It returns only a warning if the method failed.\n\nThe LDLtSolver variant use an LDLt factorization to solve the large system.\n\n\n\n\n\n","category":"function"},{"location":"reference/#FletcherPenaltySolver.update_parameters!-Tuple{Any, Any, Any}","page":"Reference","title":"FletcherPenaltySolver.update_parameters!","text":"update_parameters!(meta, sub_stp, feas)\n\nUpdate σ. If the current iterate also update ρ.\n\n\n\n\n\n","category":"method"},{"location":"reference/#FletcherPenaltySolver.update_parameters_unbdd!-Tuple{Any, Any, Any}","page":"Reference","title":"FletcherPenaltySolver.update_parameters_unbdd!","text":"update_parameters_unbdd!(meta, sub_stp, feas)\n\nStart or update δ, then call update_parameters!(meta, sub_stp, feas)\n\n\n\n\n\n","category":"method"},{"location":"#FletcherPenaltySolver.jl-Fletcher's-Penalty-Method","page":"Introduction","title":"FletcherPenaltySolver.jl - Fletcher's Penalty Method","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"FPS is a solver for equality-constrained nonlinear problems, i.e., optimization problems of the form","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":" min_x f(x) quad textst quad c(x) = 0","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"It uses other JuliaSmoothOptimizers packages for development. In particular, NLPModels.jl is used for defining the problem, and SolverCore.jl for the output. If a general inequality-constrained problem is given to the solver, it solves the problem reformulated as a SlackModel from NLPModelsModifiers.jl.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"We refer to juliasmoothoptimizers.github.io for tutorials on the NLPModel API. This framework allows the usage of models from Ampl (using AmplNLReader.jl), CUTEst (using CUTEst.jl), JuMP (using NLPModelsJuMP.jl), PDE-constrained optimization problems (using PDENLPModels.jl) and models defined with automatic differentiation (using ADNLPModels.jl).","category":"page"},{"location":"#Algorithm","page":"Introduction","title":"Algorithm","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"The function fps_solver solves a nonlinear optimization problem by iteratively solving the bound-constrained optimization problem using Fletcher penalty function:","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":" beginaligned\n min_x f(x) - c(x)^T lambda_delta(x) + fracrho2c(x)^2_2 \n mboxwhere lambda_delta(x) in textargmin_y frac12 nabla c(x)^T y - nabla f(x) ^2_2 + sigma c(x)^T y + fracdelta2y^2\n endaligned","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"For equality-constrained problems, the method iteratively solves an unconstrained problem. For bound and equality-constrained problems, the subproblems are bound-constrained problems. Any solver compatible with Stopping.jl can be used. By default, we use ipopt from NLPModelsIpopt.jl to solve the subproblem, but other solvers can be used such as knitro from NLPModelsKnitro.jl or any solvers from JSOSolvers.jl. The Stopping version of these solvers is available in StoppingInterface.jl.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"It uses LDLFactorizations.jl by default to evaluate the derivatives of the penalized subproblem, but one can also use a matrix-free version with Krylov.jl.","category":"page"},{"location":"#References","page":"Introduction","title":"References","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"Estrin, R., Friedlander, M. P., Orban, D., & Saunders, M. A. (2020). Implementing a smooth exact penalty function for equality-constrained nonlinear optimization. SIAM Journal on Scientific Computing, 42(3), A1809-A1835. 10.1137/19M1238265","category":"page"},{"location":"#How-to-Cite","page":"Introduction","title":"How to Cite","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"If you use FletcherPenaltySolver.jl in your work, please cite using the format given in CITATION.cff.","category":"page"},{"location":"#Installation","page":"Introduction","title":"Installation","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"pkg> add https://github.com/JuliaSmoothOptimizers/FletcherPenaltySolver.jl","category":"page"},{"location":"#Example","page":"Introduction","title":"Example","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"We consider in this example the minization of the Rosenbrock function.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":" min_x 100 (x₂ - x₁²)² + (x₁ - 1)²","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"The problem is modeled using ADNLPModels.jl with [-1.2; 1.0] as default initial point, and then solved using fps_solve.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"using FletcherPenaltySolver, ADNLPModels\n\n# Rosenbrock\nnlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0])\nstats = fps_solve(nlp)","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"We consider in this example the minization of the Rosenbrock function over an inequality constraint.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":" min_x 100 (x₂ - x₁²)² + (x₁ - 1)² quad textst quad 0 x₁x₂ 1","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"The problem is modeled using ADNLPModels.jl with [-1.2; 1.0] as default initial point, and then solved using fps_solve.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"using FletcherPenaltySolver, ADNLPModels\n\nnlp = ADNLPModel(\n x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2,\n [-1.2; 1.0],\n x->[x[1] * x[2]],\n [0.0],\n [1.0],\n)\nstats = fps_solve(nlp)","category":"page"},{"location":"#Bug-reports-and-discussions","page":"Introduction","title":"Bug reports and discussions","text":"","category":"section"},{"location":"","page":"Introduction","title":"Introduction","text":"If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.","category":"page"},{"location":"","page":"Introduction","title":"Introduction","text":"If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers, so questions about any of our packages are welcome.","category":"page"},{"location":"tutorial/#Tutorial","page":"Tutorial","title":"Tutorial","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Pages = [\"tutorial.md\"]","category":"page"},{"location":"tutorial/#FletcherPenaltySolver-Tutorial","page":"Tutorial","title":"FletcherPenaltySolver Tutorial","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"In this tutorial, we explore on small instances various possibilities offered by fps_solve defined in the package FletcherPenaltySolver.","category":"page"},{"location":"tutorial/#Type-stable-algorithm","page":"Tutorial","title":"Type stable algorithm","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The algorithm is implemented in pure Julia, so if one also chooses an unconstrained solver in pure Julia, we can Julia's type stability to solve optimization problems in a precision different than Float64. In the following example, we use tron from JSOSolvers.jl on a simple example in Float32.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using ADNLPModels, FletcherPenaltySolver, JSOSolvers\nT = Float32\nnlp = ADNLPModel(x -> (1 - x[1])^2, T[-1.2; 1.0], x -> [10 * (x[2] - x[1]^2)], T[0.0], T[0.0])\nstats = fps_solve(nlp, hessian_approx = Val(2), subproblem_solver = tron, rtol = T(1e-4), verbose = 1)\n(stats.dual_feas, stats.primal_feas, stats.status, typeof(stats.solution))","category":"page"},{"location":"tutorial/#A-factorization-free-solver","page":"Tutorial","title":"A factorization-free solver","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The main advantage of fps_solver is the possibility to use Hessian and Jacobian-vector products only, whenever one uses a subproblem solver with the same property. So, it is not necessary to compute and store explicitly those matrices. In the following example, we choose a problem with equality constraints from OptimizationProblems.jl.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using ADNLPModels, FletcherPenaltySolver, JSOSolvers, OptimizationProblems\nnlp = OptimizationProblems.ADNLPProblems.hs28()\nstats = fps_solve(nlp, subproblem_solver = tron, qds_solver = :iterative)\n(stats.dual_feas, stats.primal_feas, stats.status, stats.elapsed_time)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Exploring nlp's counter, we can see that no Hessian or Jacobian matrix has been evaluated.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"nlp.counters","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"We can compare this result with ipopt that uses the Jacobian and Hessian matrices.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using NLPModels, NLPModelsIpopt\nreset!(nlp);\nstats = fps_solve(nlp, subproblem_solver = ipopt, qds_solver = :iterative)\n(stats.dual_feas, stats.primal_feas, stats.status, stats.elapsed_time)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"nlp.counters","category":"page"},{"location":"tutorial/#Stopping-solver","page":"Tutorial","title":"Stopping-solver","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using ADNLPModels, FletcherPenaltySolver, Stopping\nf(x) = (x[1] - 1)^2 + 4 * (x[2] - x[1]^2)^2\nc(x) = [x[1]^2 + x[2]^2 - 2]\nT = Float64\nx0 = T[-1.2; 1.0]\nℓ, u = zeros(T, 2), 2 * ones(T, 2)\nnlp = ADNLPModel(f, x0, ℓ, u, c, zeros(1), zeros(1))\n\nstp = NLPStopping(nlp)\nstats = fps_solve(stp)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"It is then possible to explore the various quantities computed by the algorithm. For instance, recompute the gradient of the Lagrangian.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"state = stp.current_state\nstate.gx + state.Jx' * state.lambda","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Another possibility is to reuse the Stopping for another solve.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"new_x0 = 4 * ones(2)\nreinit!(stp, rstate = true, x = new_x0)\nStopping.reset!(stp.pb)\nstats = fps_solve(stp)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"We refer to Stopping.jl and https://solverstoppingjulia.github.io for tutorials and documentation.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"EditURL = \"../assets/example.jl\"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"(Image: Binder)","category":"page"},{"location":"example/#Solve-Large-Scale-Problem-with-FletcherPenaltySolver.jl","page":"Large-scale example","title":"Solve Large-Scale Problem with FletcherPenaltySolver.jl","text":"","category":"section"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"In this tutorial we use fps_solve to solve a large-scale optimization problem resulting from the discretization of a PDE-constrained optimization problem and compare the solve with Ipopt.","category":"page"},{"location":"example/#Problem-Statement","page":"Large-scale example","title":"Problem Statement","text":"","category":"section"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Let Ω = (-1,1)², we solve the following distributed Poisson control problem with Dirichlet boundary:","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":" leftlbrace\n beginaligned\n min_y in H^1_0 u in H^1 quad frac12 int_Omega y(x) - y_d(x)^2dx + fracalpha2 int_Omega u^2dx \n textst -Delta y = h + u quad x in Omega \n y = 0 quad x in partial Omega\n endaligned\n right","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"where yd(x) = -x₁² and α = 1e-2. The force term is h(x₁, x₂) = - sin(ω x₁)sin(ω x₂) with ω = π - 1/8.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"We refer to Gridap.jl for more details on modeling PDEs and PDENLPModels.jl for PDE-constrained optimization problems.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"using Gridap, PDENLPModels","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"WARNING: Method definition testargs(Gridap.Arrays.PosNegReindex{A, B} where B where A, Integer) in module Arrays at /home/runner/.julia/packages/Gridap/EZQEK/src/Arrays/PosNegReindex.jl:10 overwritten in module PDENLPModels at /home/runner/.julia/packages/PDENLPModels/pW0Iv/src/PDENLPModels.jl:16.\nERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.\nWARNING: method definition for _compute_hess_structure at /home/runner/.julia/packages/PDENLPModels/pW0Iv/src/hessian_struct_nnzh_functions.jl:70 declares type variable T but does not use it.\nWARNING: method definition for _compute_hess_structure at /home/runner/.julia/packages/PDENLPModels/pW0Iv/src/hessian_struct_nnzh_functions.jl:74 declares type variable T but does not use it.\n","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Definition of the domain and discretization","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"n = 20\ndomain = (-1, 1, -1, 1)\npartition = (n, n)\nmodel = CartesianDiscreteModel(domain, partition)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"CartesianDiscreteModel()","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Definition of the FE-spaces","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"reffe = ReferenceFE(lagrangian, Float64, 2)\nXpde = TestFESpace(model, reffe; conformity = :H1, dirichlet_tags = \"boundary\")\ny0(x) = 0.0\nYpde = TrialFESpace(Xpde, y0)\n\nreffe_con = ReferenceFE(lagrangian, Float64, 1)\nXcon = TestFESpace(model, reffe_con; conformity = :H1)\nYcon = TrialFESpace(Xcon)\nY = MultiFieldFESpace([Ypde, Ycon])","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"MultiFieldFESpace()","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Integration machinery","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"trian = Triangulation(model)\ndegree = 1\ndΩ = Measure(trian, degree)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Measure()","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Objective function","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"yd(x) = -x[1]^2\nα = 1e-2\nfunction f(y, u)\n ∫(0.5 * (yd - y) * (yd - y) + 0.5 * α * u * u) * dΩ\nend","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"f (generic function with 1 method)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Definition of the constraint operator","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"ω = π - 1 / 8\nh(x) = -sin(ω * x[1]) * sin(ω * x[2])\nfunction res(y, u, v)\n ∫(∇(v) ⊙ ∇(y) - v * u - v * h) * dΩ\nend\nop = FEOperator(res, Y, Xpde)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"FEOperatorFromWeakForm()","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Definition of the initial guess","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"npde = Gridap.FESpaces.num_free_dofs(Ypde)\nncon = Gridap.FESpaces.num_free_dofs(Ycon)\nx0 = zeros(npde + ncon);","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Overall, we built a GridapPDENLPModel, which implements the NLPModels.jl API.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"nlp = GridapPDENLPModel(x0, f, trian, Ypde, Ycon, Xpde, Xcon, op, name = \"Control elastic membrane\")\n\n(nlp.meta.nvar, nlp.meta.ncon)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"(1962, 1521)","category":"page"},{"location":"example/#Find-a-Feasible-Point","page":"Large-scale example","title":"Find a Feasible Point","text":"","category":"section"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Before solving the previously defined model, we will first improve our initial guess. We use FeasibilityResidual from NLPModelsModifiers.jl to convert the NLPModel as an NLSModel. Then, using trunk, a solver for least-squares problems implemented in JSOSolvers.jl, we find An improved guess which is close to being feasible for our large-scale problem. By default, a JSO-compliant solver such as trunk (the same applies to fps_solve) uses by default nlp.meta.x0 as an initial guess.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"using JSOSolvers, NLPModelsModifiers\n\nnls = FeasibilityResidual(nlp)\nstats_trunk = trunk(nls)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"\"Execution stats: first-order stationary\"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"We check the solution from the stats returned by trunk:","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"norm(cons(nlp, stats_trunk.solution))","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"1.6058259477686155e-5","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"We will use the solution found to initialize our solvers.","category":"page"},{"location":"example/#Solve-the-Problem","page":"Large-scale example","title":"Solve the Problem","text":"","category":"section"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Finally, we are ready to solve the PDE-constrained optimization problem with a targeted tolerance of 1e-5. In the following, we will use both Ipopt and DCI on our problem.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"using NLPModelsIpopt\n\nstats_ipopt = ipopt(nlp, x0 = stats_trunk.solution, tol = 1e-5, print_level = 0)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"\"Execution stats: first-order stationary\"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"The problem was successfully solved, and we can extract the function evaluations from the stats.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"nlp.counters","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":" Counters:\n obj: ██████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 9 grad: ███████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 10 cons: ████████████⋅⋅⋅⋅⋅⋅⋅⋅ 18 \n cons_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 cons_nln: ████████████⋅⋅⋅⋅⋅⋅⋅⋅ 18 jcon: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jgrad: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jac: ███████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 10 jac_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jac_nln: ███████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 10 jprod: ███████████████⋅⋅⋅⋅⋅ 22 jprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jprod_nln: ███████████████⋅⋅⋅⋅⋅ 22 jtprod: ████████████████████ 30 jtprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jtprod_nln: ████████████████████ 30 hess: ██████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 8 hprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jhess: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jhprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Reinitialize the counters before re-solving.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"reset!(nlp);","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"NullLogger avoids printing iteration information.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"using FletcherPenaltySolver, Logging\n\nstats_fps_solve = with_logger(NullLogger()) do\n fps_solve(nlp, stats_trunk.solution, atol = 1e-5, rtol = 1e-5)\nend","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"\"Execution stats: first-order stationary\"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"The problem was successfully solved, and we can extract the function evaluations from the stats.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"nlp.counters","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":" Counters:\n obj: ██████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 3 grad: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 cons: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 \n cons_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 cons_nln: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 jcon: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jgrad: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jac: ████████████⋅⋅⋅⋅⋅⋅⋅⋅ 6 jac_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jac_nln: ████████████⋅⋅⋅⋅⋅⋅⋅⋅ 6 jprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jprod_nln: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jtprod: ██████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5 jtprod_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n jtprod_nln: ██████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5 hess: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4 hprod: ████████████████████ 10 \n jhess: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 jhprod: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 \n","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"We now compare the two solvers with respect to the time spent,","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"stats_ipopt.elapsed_time, stats_fps_solve.elapsed_time","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"(17.819, 23.132580041885376)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"and also check objective value, feasibility and dual feasibility of ipopt and fps_solve.","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"(stats_ipopt.objective, stats_ipopt.primal_feas, stats_ipopt.dual_feas),\n(stats_fps_solve.objective, stats_fps_solve.primal_feas, stats_fps_solve.dual_feas)","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"((0.005425026428348349, 2.2204460492503135e-18, 4.2724309076258577e-7), (0.005425025968573663, 2.2204460492503135e-18, 2.968652346957424e-7))","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"Overall FletcherPenaltySolver is doing great for solving large-scale optimization problems!","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"","category":"page"},{"location":"example/","page":"Large-scale example","title":"Large-scale example","text":"This page was generated using Literate.jl.","category":"page"}] } diff --git a/previews/PR134/tutorial/index.html b/previews/PR134/tutorial/index.html index bf2633d..5e903e0 100644 --- a/previews/PR134/tutorial/index.html +++ b/previews/PR134/tutorial/index.html @@ -3,10 +3,10 @@ T = Float32 nlp = ADNLPModel(x -> (1 - x[1])^2, T[-1.2; 1.0], x -> [10 * (x[2] - x[1]^2)], T[0.0], T[0.0]) stats = fps_solve(nlp, hessian_approx = Val(2), subproblem_solver = tron, rtol = T(1e-4), verbose = 1) -(stats.dual_feas, stats.primal_feas, stats.status, typeof(stats.solution))
      (3.6239724f-6, 0.0f0, :first_order, Vector{Float32})

      A factorization-free solver

      The main advantage of fps_solver is the possibility to use Hessian and Jacobian-vector products only, whenever one uses a subproblem solver with the same property. So, it is not necessary to compute and store explicitly those matrices. In the following example, we choose a problem with equality constraints from OptimizationProblems.jl.

      using ADNLPModels, FletcherPenaltySolver, JSOSolvers, OptimizationProblems
+(stats.dual_feas, stats.primal_feas, stats.status, typeof(stats.solution))
      (0.00017044312f0, 5.9604645f-7, :first_order, Vector{Float32})

      A factorization-free solver

      The main advantage of fps_solver is the possibility to use Hessian and Jacobian-vector products only, whenever one uses a subproblem solver with the same property. So, it is not necessary to compute and store explicitly those matrices. In the following example, we choose a problem with equality constraints from OptimizationProblems.jl.

      using ADNLPModels, FletcherPenaltySolver, JSOSolvers, OptimizationProblems
 nlp = OptimizationProblems.ADNLPProblems.hs28()
 stats = fps_solve(nlp, subproblem_solver = tron, qds_solver = :iterative)
-(stats.dual_feas, stats.primal_feas, stats.status, stats.elapsed_time)
      (1.045150101384808e-9, 3.4812197569067393e-10, :first_order, 6.000412940979004)

      Exploring nlp's counter, we can see that no Hessian or Jacobian matrix has been evaluated.

      nlp.counters
        Counters:
+(stats.dual_feas, stats.primal_feas, stats.status, stats.elapsed_time)
      (1.045150101384808e-9, 3.4812197569067393e-10, :first_order, 6.0443761348724365)

      Exploring nlp's counter, we can see that no Hessian or Jacobian matrix has been evaluated.

      nlp.counters
        Counters:
              obj: █⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 4                 grad: ██⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5                 cons: ██⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5     
         cons_lin: ██⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 5             cons_nln: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                 jcon: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0     
            jgrad: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                  jac: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0              jac_lin: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0     
@@ -17,7 +17,7 @@

      We can compare this result with ipopt that uses the Jacobian and Hessian matrices.

      using NLPModels, NLPModelsIpopt
 reset!(nlp);
 stats = fps_solve(nlp, subproblem_solver = ipopt, qds_solver = :iterative)
-(stats.dual_feas, stats.primal_feas, stats.status, stats.elapsed_time)
      (4.929390229335695e-14, 0.0, :first_order, 1.0188570022583008)
      nlp.counters
        Counters:
+(stats.dual_feas, stats.primal_feas, stats.status, stats.elapsed_time)
      (4.929390229335695e-14, 0.0, :first_order, 1.0257899761199951)
      nlp.counters
        Counters:
              obj: █████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 2                 grad: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 3                 cons: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 3     
         cons_lin: ████████⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 3             cons_nln: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                 jcon: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0     
            jgrad: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                  jac: ███⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 1              jac_lin: ███⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 1     
@@ -40,4 +40,4 @@
  0.0

      Another possibility is to reuse the Stopping for another solve.

      new_x0 = 4 * ones(2)
 reinit!(stp, rstate = true, x = new_x0)
 Stopping.reset!(stp.pb)
-stats = fps_solve(stp)
      "Execution stats: first-order stationary"

      We refer to Stopping.jl and https://solverstoppingjulia.github.io for tutorials and documentation.

      +stats = fps_solve(stp)
      "Execution stats: first-order stationary"

      We refer to Stopping.jl and https://solverstoppingjulia.github.io for tutorials and documentation.