Enable delayed microservices spawning based on dependencies

[Khangf]

Description

This PR enables delayed spawning of the micrservices and megaservices from OPEA to wait for the dependencies (LLMs/microservices) to be up.

Issues

#764

Type of change

List the type of change like below. Please delete options that are not relevant.

• New feature (non-breaking change which adds new functionality)

Dependencies

N/A

Tests

We tested the code changes with the Translation example.

[louie-tsai]

instead of just waiting for LLM microservice. Should we make it more general to wait for all dependency? maybe just traverse through DAG and make sure next node available first?

[Khangf]

In fact it does wait for the other dependency if they are not LLM related. This is what we observed when we tested this:

1. Microservice will wait for LLMs to be up
2. Megaservice will wait for the microservice to be up

GenAIComps/comps/cores/mega/micro_service.py

Lines 66 to 93 in b611385

	else:
	url = f"{host}:{port}{endpoint}"
	if protocol.upper() == "HTTP":
	url = f"http://{url}"
	elif protocol.upper() == "HTTPS":
	url = f"https://{url}"
	# enable future support for gRPC?
	logger.info(f"Endpoint requested for microservice @ {url}")
	success = False
	while success is False:
	try:
	for item in methods:
	# Do we want to cater for GET request?
	# if yes, how to make it clean
	if item.upper() == "POST":
	response = requests.post(url)
	if response.status_code == 200:
	logger.info(f"Microservice endpoint is ready - {url}")
	else:
	logger.info(f"Microservice endpoint is ready - but error status code - {url}")
	success = True
	else:
	logger.info(f"Microservice endpoint method not supported - Skipping - {url}")
	success = True
	except requests.exceptions.RequestException as e:
	logger.info(f"Error: {e} - {url}")
	time.sleep(2.5)