問題

當您添加配置設置時，在Apache中輸入它火花配置文本區域，新設置將替換現有設置，而不是被追加。

版本

Databricks運行時5.1及以下版本。

導致

類中創建的配置文件中讀取設置集群UI，並覆蓋默認設置。

例如，當您添加以下內容時extraJavaOptions到火花配置文本區域:

spark.executor.extraJavaOptions—javaagent:/opt/prometheus_jmx_export /jmx_prometheus_javaagent.jar=9404:/opt/prometheus _jmx_export /jmx_prometheus_javaagent.yml

然後,在火花UI>環境>火花屬性下spark.executor.extraJavaOptions，僅新增配置設置顯示:

- javaagent: / opt / prometheus_jmx_exporter / jmx_prometheus_javaagent.jar = 9404: / opt /普羅米修斯_jmx_exporter / jmx_prometheus_javaagent.yml

刪除所有現有設置。

作為參考，默認設置如下:

-Djava.io。tmpdir=/local_disk0/tmp -XX:ReservedCodeCacheSize=256m -XX: +UseCodeCacheFlushing - ddatabricks . tmpdir=/local_disk0/tmp -XX:ReservedCodeCacheSize=256m -serviceName=spark-executor-1 - Djava.security.properties=/databricks/spark/dbconf/java/extra. serviceName=spark-executor-1 - Djava.security.properties=/databricks/spark/dbconf/java/extra. propertiessecurity -XX:+PrintFlagsFinal -XX:+ PrintGCDateStamps -verbose:gc -XX:+PrintGCDetails - xss4m - djavax.xml . datatypefactory =com. sun.org.apache.xerces.internal.jaxp.datatypefactoryimpl - djavax.xml.parser.documentbuilderfactory =com.sun.org.apache.xerces.internal.jaxp.SAXParserFact oryImpl - djavax.xml.parser.saxparserfactory =com.sun.org.apache.xerces.internal.jaxp.SAXParserFact oryImpl - djavax.xml.parser.xml.validation.schemafactory =https://www.w3.org/2001/XMLSchema=com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory—Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOMX SImplementationSourceImpl

解決方案

將新配置設置添加到spark.executor.extraJavaOptions在不丟失默認設置的情況下:

1. 在火花UI>環境>火花屬性，選擇並複製默認設置的所有屬性spark.executor.extraJavaOptions．
2. 點擊編輯．
3. 在火花配置文字區(集群>集群名稱>高級選項>火花)，粘貼默認設置。
4. 將新的配置設置追加到默認設置下麵。
5. 在文本區域外單擊，然後單擊確認．
6. 重新啟動集群。

例如，假設將以下設置粘貼到火花配置文本區域。新的配置設置被追加到默認設置中。

spark.executor.extraJavaOptions = -Djava.io。tmpdir=/local_disk0/tmp -XX: ReservedCodeCacheSize=256m -XX:+UseCodeCacheFlushing - ddatabricks . tmpdir=/local_disk0/tmp -XX: ReservedCodeCacheSize=256m -XX:+UseCodeCacheFlushing - ddatabricks . tmpdir=/local_disk0/tmpserviceName=spark- execuator -1 - djava .security.properties=/databricks/spark/dbconf/java/extra. serviceName=spark- execuator -1 - djava .security.properties=/databricks/spark/dbconf/java/extra. propertiessecurity -XX:+ PrintFlagsFinal -XX:+PrintGCDateStamps -verbose:gc -XX:+PrintGCDetails - xss4m - djavax.xml . datatypefactory =com. sun.org.apache.xerces.internal.jaxp.datatypefactoryimpl - djavax.xml.parser.documentbuilderfactory =com.sun.org.apache.xerces.internal.jaxp.DocumentB uilderFactoryImpl - Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactor yImpl - djavax.xml.parsers.schemafactory:https://www.w3.org/2001/XMLSchema=com.sun.org.apache.xerxmlschemafactory—Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser—Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOMXSImplem entationSourceImpl—javaagent:/opt/prometheus_jmx_export /jmx_prometheus_javaagent.jar=9404:/opt/prometheus_jm x_export /jmx_prometheus_javaagent.yml

重新啟動集群後，將顯示默認設置和新添加的配置設置火花UI>環境>火花屬性．