Is there a way to tell Pyright to dump LSIF data for an entire workspace?

[vberlier]

I want to make a Sphinx extension that adds hover text to code examples like what twoslash does for typescript code samples. I'd like to use an LSIF dump to get very precise type information and I think Pyright would be the most suitable tool for this since it already bundles a language server.

I first tried to look for something generic that can query any language server to produce an LSIF dump but I haven't found anything promising. That's why I'm wondering if at least there's a way to do this for Pyright specifically.

I also have other potential use-cases for the LSIF dump such as finding references to documented functions/classes in example files and displaying them (shamelessly inspired by https://twitter.com/wcrichton/status/1456112165744615426).

[erictraut]

As you mentioned, pyright is a language server, and it already provides hover text. It works with other language clients, so if you have an LSP-compatible language client, it should work.

We don't currently have any plans to add support for generating LSIF dumps, nor would we want to maintain such functionality as part of the pyright code base. That said, pyright is open-source, so you could fork it and use it as the basis for a tool that generates LSIF output. The closest thing that pyright offers is the ability to create type stubs from a code base. You could use the createTypeStubs.ts module as a template for how to do this.

It looks like there is at least one open-source project that generates LSIF dumps from a Python code base: https://github.com/sourcegraph/lsif-py.

[vberlier]

Hmm I see. The project you mentioned does generate LSIF dumps for Python but it's hardcoded to use Jedi, which is nowhere near as accurate as Pyright. Also it doesn't interface with Jedi through the language server protocol. It calls the internal APIs directly so it can't be easily adapted to work with Pyright. I'm going to look into how createTypeStubs.ts works, but now I'm also considering trying to build the thing I wish existed in the first place: a generic utility to create LSIF dumps by querying arbitrary language servers. Oh boy wish me luck.

[Strum355]

Hi @vberlier, Im a member of the Sourcegraph Code Intelligence team 🙂 just want to let you know that we are planning to revisit https://github.com/sourcegraph/lsif-py in the somewhat near future, potentially using Pyright. We have worked on two PoC's for the sake of comparison, one using Pyright and one using Jedi but with much improved results compared to the existing lsif-py, but have not had the time to create fully featured & released indexers out of either due to other priorities at the time.

a generic utility to create LSIF dumps by querying arbitrary language servers

while this could work in theory, the performance is most likely going to be highly non-optimal. You can find some discussion Ive had here #1928, which should explain the why.

[vberlier]

one using Pyright and one using Jedi but with much improved results compared to the existing lsif-py

Oh nice! Looking forward to using this then. Also yeah I don't expect a generic LSIF utility to be very efficient but it would open up LSIF to a lot of languages for free. I'll read through the discussion, looks very interesting!