Automatically connect to new dashboard #26
Comments
|
@dhirschfeld, thanks for the report. My intention was that the "search" icon next to the URL would attempt to automatically query the dashboard URL from the currently active notebook/console. Is it not working for your use-case? We may need to harden that a bit. Or are the panes not correctly loading a valid URL? If so, can you provide a sequence of commands so I can try to reproduce (ideally with a local cluster)? |
|
@ian-r-rose I'm having the same issue. Not sure what the search icon is doing, but it's not getting the correct ip. This is what happens after i restart the kernel and run everything up to the client part in cell [4]. It would be nice if this problem can get fixed, that way i can ''reload'' the current open windows, after restarting the kernel. Without having to re-open them and creating the layout i like to have.
TLDR: the search icon makes it get 'http://192.168.0.100/20840/1:8787/' while it should get 'http://localhost:8787/status'. |
|
It looks like this is a problem within the dask/distributed library. The This can be fixed here: Probably we want to replace host with @deKeijzer is this something that you have the time to contribute? also cc @jacobtomlinson |
|
@mrocklin Sadly not at the moment, but thanks for asking. I'm still quite busy with actually learning Dask haha. |
|
Well then short term I recommend setting Client(processes=True) or just
putting http://localhost:8787 into the search bar manually.
…On Mon, Oct 1, 2018 at 11:27 AM Brian de Keijzer ***@***.***> wrote:
@mrocklin <https://github.com/mrocklin> Sadly not at the moment, but
thanks for asking. I'm still quiet busy with actually learning Dask haha.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#26 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AASszNBkb8SQBhGaASh4mnXqJiG-ujzCks5ugjRQgaJpZM4W17u2>
.
|
What I'm observing is that when I close a dask cluster I see the The only way to get the panes to recognise the new url is to close and then re-open them from the sidebar. This necessitates a lot of mucking about rearranging the layout you had previously so it would be great if the panes automatically refreshed the connection when you entered a new url. This sounds like a different problem to what @deKeijzer is observing as my url is correct and the extension is connected (buttons are orange) - it's just the existing panes aren't recognising a new url. |
|
@dhirschfeld I am having a tough time reproducing this. I have a notebook with a single cell: from dask.distributed import Client
client = Client()
clientand am performing the following steps:
What browser are you using? I can reproduce this behavior in both firefox and chrome on Ubuntu. |
You mean you see the same I'm using chrome. My distributed scheduler is remote and every time I restart it gets mapped to a new port - that could be the key here. I'll see if I can test further to narrow it down... |
|
This should be fixed by #36, once that lands. |
|
Any updates on this issue? I am experiencing similar greyed out buttons to access the dashboard and receive a connection refused when opening 'http://localhost:8888/proxy/8787/status' in a new tab in chrome. Likewise, I use a PBS cluster and pangeo on cheyenne and port forward to see the lab notebook running remotely. I was once able to connect to a client and start some workers but also saw the 404: Not Found for the dask task stream. Would love to get this up and running to figure out my worker load balance and see the task stream.
|
|
Chiming in here to say that I'm also having this problem, using dask-labextension on a slurm cluster behind a vpn. I can access the dashboard through an ssh tunnel, but putting the tunneled dashboard url ( |
|
@bmcfee are you also accessing Jupyter at Just to check you can view the dashboard at |
|
No, our jupyter setup runs a bit differently, and is not exposed as localhost. Otherwise, that's correct: I can tunnel a port on localhost to the machine running the dask dashboard, visit the dashboard directly, but pasting the link doesn't work. |
This could be the problem then. If you remove |
Aha, that does seem to work! Is there any way (eg from a cell magic or something) to push that over to labextension from a notebook cell? (Background: I'm using this for teaching purposes, and it'd be best if students didn't have to copy-mangle-paste to get the dashboard working.) |
|
Actually, I spoke too soon -- it doesn't quite work. Modifying the url as you suggest does connect to the dashboard, and turns the links orange, but none of the dashboard contents show up. However, if I connect to the dashboard through the ssh tunnel, everything does appear as expected. This might be down to a quirk of how our network is set up, in which case, feel free to ignore this. Seems strange though! |
You can set the dashboard url templarte in your Dask config which is picked up by the Lab extension.
Does the browser developer tools show any error relating to the plot contents? Are you able to right click the plot tabs and open in a new browser tab and see it working there?
This likely is the case, but it would be nice to get to the bottom of what is causing it. |
Loading the proxy url directly does bring up the page, but it's generally blank (except for the |
|
Interesting, we've seen similar issues in a few other places where whatever proxy is being placed in front of Jupyter is not correctly handling websocket connections.
Are you able to share any more detail on how you are accessing Jupyter? Specifically what proxying technology is being used. Am I correct in thinking the regular dashboard works via the |
We use Open OnDemand for this, but I don't know much more than that, sorry!
That's right -- the menu at the top shows up, but all tabs (except |
Ok it sounds like all the websocket connections are being dropped, which is how we update the plots. I suspect that Open OnDemand is proxying the HTTP traffic and perhaps not handling the websocket connections correctly. Jupyter also uses websockets, so things must be working correctly in that case. I'm not sure what is going wrong here though. We've seen a similar problem in dask/dask#5432 on AWS Sagemaker. |
2.3.0. Here's a full conda env |
|
Thanks @bmcfee that rule it out as a 1.x issue. My worry here is that it is an issue being caused by the Open OnDemand proxy which we have no control over. |
A common workflow for me when using distributed is to layout my JupyterLab with a couple of
daskpanes so I can keep an eye on the cluster. I'm often starting new clusters and pasting new/statusurls in thedask-labextensionsidebar. When I paste in a new url the currentdaskpanes show a 404 message and AFAICS the only way to get them to recognise the new connection is to close the existing panes, open new panes and then recreate the layout as I had it previously.It would be great if the
daskpanes could recognise a new url and connect to it automatically. If there's a reason that can't work automatically, perhaps rather than a 404 message the panes could have a refresh button which would allow them to connect to a new url?The text was updated successfully, but these errors were encountered: