Skip to content

Commit

Permalink
[SPARK-48312][SQL] Improve Alias.removeNonInheritableMetadata perform…
Browse files Browse the repository at this point in the history
…ance

### What changes were proposed in this pull request?
Improve `Alias.removeNonInheritableMetadata` performance - avoid using `MetadataBuilder` when there is no metadata or when there are no non-inheritable metadata keys to remove.

### Why are the changes needed?
In case of wide VIEWs with many Aliases this method slows down the analysis

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Existing tests

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#46622 from vladimirg-db/vladimirg-db/improve-remove-non-inheritable-metadata-performance.

Authored-by: Vladimir Golubev <vladimir.golubev@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
  • Loading branch information
vladimirg-db authored and cloud-fan committed May 17, 2024
1 parent 15fb478 commit 3edd6c7
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,13 @@ sealed class Metadata private[types] (private[types] val map: Map[String, Any])
/** Tests whether this Metadata contains a binding for a key. */
def contains(key: String): Boolean = map.contains(key)

/**
* Tests whether this Metadata is empty.
*
* @since 4.0.0
*/
def isEmpty: Boolean = map.isEmpty

/** Gets a Long. */
def getLong(key: String): Long = get(key)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -205,10 +205,18 @@ case class Alias(child: Expression, name: String)(
""
}

/**
* This function is performance-sensitive, so we should avoid `MetadataBuilder` manipulation,
* because it performs heavy operations on maps
*/
private def removeNonInheritableMetadata(metadata: Metadata): Metadata = {
val builder = new MetadataBuilder().withMetadata(metadata)
nonInheritableMetadataKeys.foreach(builder.remove)
builder.build()
if (metadata.isEmpty || nonInheritableMetadataKeys.forall(!metadata.contains(_))) {
metadata
} else {
val builder = new MetadataBuilder().withMetadata(metadata)
nonInheritableMetadataKeys.foreach(builder.remove)
builder.build()
}
}

override def toString: String = s"$child AS $name#${exprId.id}$typeSuffix$delaySuffix"
Expand Down

0 comments on commit 3edd6c7

Please sign in to comment.