[DigitalOcean] droplet integration

[asaiacai]

This adds digital ocean droplets to the sky.

Tested (run the relevant ones):

• Code formatting: bash format.sh
• All smoke tests: pytest tests/test_smoke.py --do

[moinnadeem]

@asaiacai I would be really interested in using this work -- do you know when you might be able to land this PR?

[asaiacai]

Hello! I’m aiming for this to land in next two weeks providing testing goes smoothly. Stay tuned!

[asaiacai]

@Michaelvll i think this PR is finally ready, once you have a free moment! I've currently disabled tests for gpu droplets as they are still in early access, but I think I got the remaining tests to work on normal CPU droplets fine. Let me know if you find any issues.

[asaiacai]

also this catalog update is required to pass the sky serve requests since I added cheaper 2vcpu instances.

skypilot-org/skypilot-catalog#83

[cblmemo]

Thanks for this amazing work @asaiacai ! This would be very helpful. Left some discussions ;)

[cblmemo]

Why don't we support it? I saw relevant code snippet in the ray yaml file.

[asaiacai]

I included a startup script that installs docker, but ssh comes up in parallel before the startup script finishes running resulting in the docker environment setup failing. Docker does eventually get setup on the machine. If I for example, reattempt to launch on the same cluster after waiting for like 10 seconds, I can successfully use the docker environment, but the times i tested it never worked on freshly provisioned instances. Let me know what would make sense here.

[cblmemo]

Is it possible to use a base image with docker installed? Or, can we wait for the docker to finish setup for DO cloud?

skypilot/sky/provision/docker_utils.py

Lines 336 to 343 in e870839

	def _check_docker_installed(self):
	no_exist = 'NoExist'
	cleaned_output = self._run(
	f'command -v {self.docker_cmd} || echo {no_exist!r}')
	if no_exist in cleaned_output or 'docker' not in cleaned_output:
	logger.error(
	f'{self.docker_cmd.capitalize()} not installed. Please use an '
	f'image with {self.docker_cmd.capitalize()} installed.')

[asaiacai]

i added waiting for docker to _check_docker_installed but let me know if this should only be specific to DO

[cblmemo]

My concern is that it will take too long to error out for an image without docker installation. Is it possible that we somehow detect if the docker installation is in progress, and immediately return if we found no installation?

[asaiacai]

@cblmemo reran smoke, but if you have a moment to review again, thanks in advance!