Skip to content

Commit

Permalink
[SPARK-50021][CORE][UI][3.5] Fix ApplicationPage to hide App UI lin…
Browse files Browse the repository at this point in the history
…ks when UI is disabled

### What changes were proposed in this pull request?

This PR aims to fix `ApplicationPage` to hide UI link when UI is disabled.

### Why are the changes needed?

Previously, Spark throws `HTTP ERROR 500 java.lang.IllegalArgumentException: Invalid URI host: null (authority: null)` like the following

**1. PREPARATION**
```
$ cat conf/spark-defaults.conf
spark.ui.reverseProxy true
spark.ui.reverseProxyUrl http://localhost:8080

$ sbin/start-master.sh

$ sbin/start-worker.sh spark://$(hostname):7077

$ bin/spark-shell --master spark://$(hostname):7077 -c spark.ui.enabled=false
```

**2. BEFORE**
<img width="496" alt="Screenshot 2024-10-17 at 21 24 32" src="https://github.com/user-attachments/assets/9884790c-a294-4e61-b630-7758c5532afc">

<img width="1002" alt="Screenshot 2024-10-17 at 21 24 51" src="https://github.com/user-attachments/assets/f1e3a121-37ba-4525-a433-21ad15402edf">

**3. AFTER**
<img width="493" alt="Screenshot 2024-10-17 at 21 22 26" src="https://github.com/user-attachments/assets/7a1ef578-3d9f-495e-9545-6edd26b4d565">

### Does this PR introduce _any_ user-facing change?

Yes, but previously it was a broken link.

### How was this patch tested?

Pass the CIs with the newly added test case.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#48547 from dongjoon-hyun/SPARK-50021-3.5.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
dongjoon-hyun committed Oct 18, 2024
1 parent 6b9b3c0 commit 3a4ebae
Show file tree
Hide file tree
Showing 2 changed files with 83 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ import javax.servlet.http.HttpServletRequest

import scala.xml.Node

import org.apache.commons.lang3.StringUtils

import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, RequestMasterState}
import org.apache.spark.deploy.ExecutorState
import org.apache.spark.deploy.StandaloneResourceUtils.{formatResourceRequirements, formatResourcesAddresses}
Expand Down Expand Up @@ -93,10 +95,14 @@ private[ui] class ApplicationPage(parent: MasterWebUI) extends WebUIPage("app")
<li><strong>State:</strong> {app.state}</li>
{
if (!app.isFinished) {
<li><strong>
<a href={UIUtils.makeHref(parent.master.reverseProxy,
app.id, app.desc.appUiUrl)}>Application Detail UI</a>
</strong></li>
if (StringUtils.isBlank(app.desc.appUiUrl)) {
<li><strong>Application UI:</strong> Disabled</li>
} else {
<li><strong>
<a href={UIUtils.makeHref(parent.master.reverseProxy,
app.id, app.desc.appUiUrl)}>Application Detail UI</a>
</strong></li>
}
}
}
</ul>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.spark.deploy.master.ui

import java.util.Date
import javax.servlet.http.HttpServletRequest

import org.mockito.Mockito.{mock, when}

import org.apache.spark.SparkFunSuite
import org.apache.spark.deploy.ApplicationDescription
import org.apache.spark.deploy.DeployMessages.{MasterStateResponse, RequestMasterState}
import org.apache.spark.deploy.master.{ApplicationInfo, ApplicationState, Master}
import org.apache.spark.resource.ResourceProfile
import org.apache.spark.rpc.RpcEndpointRef

class ApplicationPageSuite extends SparkFunSuite {

private val master = mock(classOf[Master])

private val rp = new ResourceProfile(Map.empty, Map.empty)
private val desc = ApplicationDescription("name", Some(4), null, "appUiUrl", rp)
private val descWithoutUI = ApplicationDescription("name", Some(4), null, "", rp)
private val appFinished = new ApplicationInfo(0, "app-finished", desc, new Date, null, 1)
appFinished.markFinished(ApplicationState.FINISHED)
private val appLive = new ApplicationInfo(0, "app-live", desc, new Date, null, 1)
private val appLiveWithoutUI =
new ApplicationInfo(0, "app-live-without-ui", descWithoutUI, new Date, null, 1)

private val state = mock(classOf[MasterStateResponse])
when(state.completedApps).thenReturn(Array(appFinished))
when(state.activeApps).thenReturn(Array(appLive, appLiveWithoutUI))

private val rpc = mock(classOf[RpcEndpointRef])
when(rpc.askSync[MasterStateResponse](RequestMasterState)).thenReturn(state)

private val masterWebUI = mock(classOf[MasterWebUI])
when(masterWebUI.master).thenReturn(master)
when(masterWebUI.masterEndpointRef).thenReturn(rpc)

test("SPARK-45774: Application Detail UI") {
val request = mock(classOf[HttpServletRequest])
when(request.getParameter("appId")).thenReturn("app-live")

val result = new ApplicationPage(masterWebUI).render(request).toString()
assert(result.contains("Application Detail UI"))
assert(!result.contains("Application History UI"))
}

test("SPARK-50021: Application Detail UI is empty when spark.ui.enabled=false") {
val request = mock(classOf[HttpServletRequest])
when(request.getParameter("appId")).thenReturn("app-live-without-ui")

val result = new ApplicationPage(masterWebUI).render(request).toString()
assert(result.contains("Application UI:</strong> Disabled"))
assert(!result.contains("Application History UI"))
}
}

0 comments on commit 3a4ebae

Please sign in to comment.