build based on e242770

previews/PR64/examples/0_intro/index.html

@@ -25,4 +25,4 @@
     return c
 end</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">conditions (generic function with 1 method)</code></pre><p>Finally, we construct a wrapper <code>implicit</code> around the previous objects. By default, <code>forward</code> is assumed to return a single output and <code>conditions</code> is assumed to accept 2 arguments.</p><pre><code class="language-julia hljs">implicit = ImplicitFunction(forward, conditions)</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">ImplicitFunction(forward, conditions, IterativeLinearSolver())</code></pre><p>What does this wrapper do? When we call it as a function, it just falls back on <code>first ∘ implicit.forward</code>, so unsurprisingly we get the first output <span>$y(x)$</span>.</p><pre><code class="language-julia hljs">implicit(x) ≈ sqrt.(x)</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">true</code></pre><p>And when we try to compute its Jacobian, the <a href="https://en.wikipedia.org/wiki/Implicit_function_theorem">implicit function theorem</a> is applied in the background to circumvent the lack of differentiablility of the forward mapping.</p><h2 id="Forward-and-reverse-mode-autodiff"><a class="docs-heading-anchor" href="#Forward-and-reverse-mode-autodiff">Forward and reverse mode autodiff</a><a id="Forward-and-reverse-mode-autodiff-1"></a><a class="docs-heading-anchor-permalink" href="#Forward-and-reverse-mode-autodiff" title="Permalink"></a></h2><p>Now ForwardDiff.jl works seamlessly.</p><pre><code class="language-julia hljs">ForwardDiff.jacobian(implicit, x) ≈ J</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">true</code></pre><p>And so does Zygote.jl. Hurray!</p><pre><code class="language-julia hljs">Zygote.jacobian(implicit, x)[1] ≈ J</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">true</code></pre><h2 id="Second-derivative"><a class="docs-heading-anchor" href="#Second-derivative">Second derivative</a><a id="Second-derivative-1"></a><a class="docs-heading-anchor-permalink" href="#Second-derivative" title="Permalink"></a></h2><p>We can even go higher-order by mixing the two packages (forward-over-reverse mode). The only technical requirement is to switch the linear solver to something that can handle dual numbers:</p><pre><code class="language-julia hljs">implicit_higher_order = ImplicitFunction(forward, conditions, DirectLinearSolver())</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">ImplicitFunction(forward, conditions, DirectLinearSolver())</code></pre><p>Then the Jacobian itself is differentiable.</p><pre><code class="language-julia hljs">h = rand(2)
 J_Z(t) = Zygote.jacobian(implicit_higher_order, x .+ t .* h)[1]
-ForwardDiff.derivative(J_Z, 0) ≈ Diagonal((-0.25 .* h) ./ (x .^ 1.5))</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">true</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../../">« Home</a><a class="docs-footer-nextpage" href="../1_basic/">Basic use cases »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Sunday 30 July 2023 12:47">Sunday 30 July 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+ForwardDiff.derivative(J_Z, 0) ≈ Diagonal((-0.25 .* h) ./ (x .^ 1.5))</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">true</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../../">« Home</a><a class="docs-footer-nextpage" href="../1_basic/">Basic use cases »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Sunday 30 July 2023 13:05">Sunday 30 July 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR64/examples/1_basic/index.html

@@ -71,4 +71,4 @@
     Zygote.jacobian(_x -> forward_fixedpoint(_x; iterations=10), x)[1]
 catch e
     e
-end</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">ErrorException(&quot;Mutating arrays is not supported -- called copyto!(Vector{Float64}, ...)\nThis error occurs when you ask Zygote to differentiate operations that change\nthe elements of arrays in place (e.g. setting values with x .= ...)\n\nPossible fixes:\n- avoid mutating operations (preferred)\n- or read the documentation and solutions for this error\n  https://fluxml.ai/Zygote.jl/latest/limitations\n&quot;)</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../0_intro/">« Introduction</a><a class="docs-footer-nextpage" href="../2_advanced/">Advanced use cases »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Sunday 30 July 2023 12:47">Sunday 30 July 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+end</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">ErrorException(&quot;Mutating arrays is not supported -- called copyto!(Vector{Float64}, ...)\nThis error occurs when you ask Zygote to differentiate operations that change\nthe elements of arrays in place (e.g. setting values with x .= ...)\n\nPossible fixes:\n- avoid mutating operations (preferred)\n- or read the documentation and solutions for this error\n  https://fluxml.ai/Zygote.jl/latest/limitations\n&quot;)</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../0_intro/">« Introduction</a><a class="docs-footer-nextpage" href="../2_advanced/">Advanced use cases »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Sunday 30 July 2023 13:06">Sunday 30 July 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR64/examples/2_advanced/index.html

@@ -25,8 +25,8 @@
 end</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">conditions_cstr_optim (generic function with 1 method)</code></pre><p>We now have all the ingredients to construct our implicit function.</p><pre><code class="language-julia hljs">implicit_cstr_optim = ImplicitFunction(forward_cstr_optim, conditions_cstr_optim)</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">ImplicitFunction(forward_cstr_optim, conditions_cstr_optim, IterativeLinearSolver())</code></pre><p>And indeed, it behaves as it should when we call it:</p><pre><code class="language-julia hljs">x = rand(2) .+ [0, 1]</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">2-element Vector{Float64}:
  0.22442135286865494
  1.3267275094228514</code></pre><p>The second component of <span>$x$</span> is <span>$&gt; 1$</span>, so its square root will be thresholded to one, and the corresponding derivative will be <span>$0$</span>.</p><pre><code class="language-julia hljs">implicit_cstr_optim(x) .^ 2</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">2-element Vector{Float64}:
- 0.22442135286146742
- 0.9999999995782778</code></pre><pre><code class="language-julia hljs">J_thres = Diagonal([0.5 / sqrt(x[1]), 0])</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">2×2 LinearAlgebra.Diagonal{Float64, Vector{Float64}}:
+ 0.22442135289492518
+ 0.999999999578276</code></pre><pre><code class="language-julia hljs">J_thres = Diagonal([0.5 / sqrt(x[1]), 0])</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">2×2 LinearAlgebra.Diagonal{Float64, Vector{Float64}}:
  1.05545   ⋅ 
   ⋅       0.0</code></pre><p>Forward mode autodiff</p><pre><code class="language-julia hljs">ForwardDiff.jacobian(implicit_cstr_optim, x)</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">2×2 Matrix{Float64}:
   1.05545  -0.0
@@ -38,4 +38,4 @@
     Zygote.jacobian(forward_cstr_optim, x)[1]
 catch e
     e
-end</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">Zygote.CompileError(Tuple{typeof(Optim.optimize), NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Optim.Fminbox{Optim.GradientDescent{LineSearches.InitialPrevious{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Optim.var&quot;#13#15&quot;}, Float64, Optim.var&quot;#49#51&quot;}, Optim.Options{Float64, Nothing}}, ErrorException(&quot;try/catch is not supported.\nRefer to the Zygote documentation for fixes.\nhttps://fluxml.ai/Zygote.jl/latest/limitations\n&quot;))</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../1_basic/">« Basic use cases</a><a class="docs-footer-nextpage" href="../3_tricks/">Tricks »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Sunday 30 July 2023 12:47">Sunday 30 July 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+end</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">Zygote.CompileError(Tuple{typeof(Optim.optimize), NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Optim.Fminbox{Optim.GradientDescent{LineSearches.InitialPrevious{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Optim.var&quot;#13#15&quot;}, Float64, Optim.var&quot;#49#51&quot;}, Optim.Options{Float64, Nothing}}, ErrorException(&quot;try/catch is not supported.\nRefer to the Zygote documentation for fixes.\nhttps://fluxml.ai/Zygote.jl/latest/limitations\n&quot;))</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../1_basic/">« Basic use cases</a><a class="docs-footer-nextpage" href="../3_tricks/">Tricks »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Sunday 30 July 2023 13:06">Sunday 30 July 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR64/examples/3_tricks/index.html

@@ -44,4 +44,4 @@
 
 implicit_cointoss(x)</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">2-element Vector{Float64}:
  2.0
- 2.0</code></pre><p>Or if you also need the byproduct, you can do</p><pre><code class="language-julia hljs">implicit_cointoss(x, ReturnByproduct())</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">([0.5, 0.5], false)</code></pre><p>But whatever you choose, the byproduct is taken into account during differentiation!</p><pre><code class="language-julia hljs">Zygote.withjacobian(implicit_cointoss, x)</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">(val = [0.5, 0.5], grad = ([0.5 0.0; 0.0 0.5],))</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../2_advanced/">« Advanced use cases</a><a class="docs-footer-nextpage" href="../../api/">API reference »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Sunday 30 July 2023 12:47">Sunday 30 July 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+ 2.0</code></pre><p>Or if you also need the byproduct, you can do</p><pre><code class="language-julia hljs">implicit_cointoss(x, ReturnByproduct())</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">([0.5, 0.5], false)</code></pre><p>But whatever you choose, the byproduct is taken into account during differentiation!</p><pre><code class="language-julia hljs">Zygote.withjacobian(implicit_cointoss, x)</code></pre><pre class="documenter-example-output"><code class="nohighlight hljs ansi">(val = [0.5, 0.5], grad = ([0.5 0.0; 0.0 0.5],))</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../2_advanced/">« Advanced use cases</a><a class="docs-footer-nextpage" href="../../api/">API reference »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Sunday 30 July 2023 13:05">Sunday 30 July 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>