Access to IModelParamsExtensions

[saddam213]

Can we set this class as public, need access to this method for building multiple contexts outside LLamaSharp

I stood up a wee demo of the web UI too :)
https://llamaweb.chainstack.nz/

[martindevans]

Hopefully it won't be needed public long term, but for now this makes sense 👍

[saddam213]

No it wont, once LLamaSharp have multiple context support it can be all locked down

I think @AsakusaRinne was looking at supporting this via the executor classes, that is a way to big of a change for a non project owner, so I have created a shim project to enable multiple context so I can continue with the UI projects as they all rely on more than 1 context

[martindevans]

I'm playing around with some ideas for how best to expose it as well. At the moment I have a (half built) demo scene with the LLM talking to itself using 2 contexts. My plan is to build that out and improve whatever APIs I need to do it.

Do you have a link to your shim project? Might provide me some helpful pointers on how to achieve it :)

[saddam213]

My shim is just the implemetation of the PR I made las week

I just splits LLamaModel into LLamaModel + LLamaContext

The shim project is LLamaHub it just copies a bunch of classes accross to allow the minimum I need to continue

Its pretty much tthe same implemetation as PR 76

[martindevans]

The demo is very cool by the way! Just checked it out.

[saddam213]

Each session is its own context, so you can open multple tabs, set differnt parameters and compare the output

way easier than using the commanline, there is also a Tag package there if you want to run it locally