集群CLI
通過將Databricks集群的CLI子命令追加到磚集群．
數據庫集群-h
用法:databricks clusters [OPTIONS] COMMAND [ARGS]…與Databricks集群交互的實用程序。選項:-v，——version [version] -h，——help顯示此信息並退出。命令:create創建Databricks集群。選項:——JSON - File PATH POST到/api/2.0/clusters/create的JSON請求文件。POST到/api/2.0/clusters/create的json字符串。delete刪除Databricks集群。選項:——cluster-id CLUSTER_ID可以在https://< databics -instance>/?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration中找到。edit編輯Databricks集群。選項:——JSON - File PATH POST到/api/2.0/clusters/edit的JSON請求文件。 --json JSON JSON string to POST to /api/2.0/clusters/edit. events Gets events for a Spark cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///#/setting/clusters/$CLUSTER_ID/configuration. [required] --start-time TEXT The start time in epoch milliseconds. If unprovided, returns events starting from the beginning of time. --end-time TEXT The end time in epoch milliseconds. If unprovided, returns events up to the current time --order TEXT The order to list events in; either ASC or DESC. Defaults to DESC (most recent first). --event-type TEXT An event types to filter on (specify multiple event types by passing the --event-type option multiple times). If empty, all event types are returned. --offset TEXT The offset in the result set. Defaults to 0 (no offset). When an offset is specified and the results are requested in descending order, the end_time field is required. --limit TEXT The maximum number of events to include in a page of events. Defaults to 50, and maximum allowed value is 500. --output FORMAT can be "JSON" or "TABLE". Set to TABLE by default. get Retrieves metadata about a cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. list Lists active and recently terminated clusters. Options: --output FORMAT JSON or TABLE. Set to TABLE by default. list-node-types Lists node types for a cluster. list-zones Lists zones where clusters can be created. permanent-delete Permanently deletes a cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. resize Resizes a Databricks cluster given its ID. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. --num-workers INTEGER Number of workers. [required] restart Restarts a Databricks cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. spark-versions Lists possible Databricks Runtime versions. start Starts a terminated Databricks cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration.
創建集群
要顯示使用文檔，請運行磚集群創建——幫助．
數據集群創建——json-file create-cluster.json
create-cluster.json：
｛“cluster_name”：“my-cluster”，“spark_version”：“7.3.x-scala2.12”，“node_type_id”：“i3.xlarge”，“spark_conf”：｛“spark.speculation”：真正的}，“aws_attributes”：｛“可用性”：“現貨”，“zone_id”：“us-west-2a”}，“num_workers”：25｝
｛“cluster_id”:“1234 - 567890 batch123”｝
更改集群配置
要顯示使用文檔，請運行磚集群編輯——幫助．
數據庫集群編輯——json-file edit-cluster.json
edit-cluster.json：
｛“cluster_id”：“1234 - 567890 batch123”，“num_workers”：10，“spark_version”：“7.3.x-scala2.12”，“node_type_id”：“i3.xlarge”｝
如果成功，則無輸出。
列出集群的事件
要顯示使用文檔，請運行磚集群事件——幫助．
數據集群事件＼——cluster-id1234567890 - batch123＼——啟動時間1617238800000＼——世界末日1619485200000＼——訂單DESC＼——限製5＼——事件類型的運行＼——JSON輸出＼|金橋。
｛“事件”:[｛“cluster_id”:“1234 - 567890 batch123”,“時間戳”:1619214150232,“類型”:“運行”,"詳細信息":{“current_num_workers”:2“target_num_workers”:2｝}，.．.｛“cluster_id”:“1234 - 567890 batch123”,“時間戳”:1617895221986,“類型”:“運行”,"詳細信息":{“current_num_workers”:2“target_num_workers”:2｝｝)," next_page ": {“cluster_id”:“1234 - 567890 batch123”,“start_time”:1617238800000,“end_time”:1619485200000,“訂單”:“DESC”,“event_types”:(“運行”),“抵消”:5“限製”:5}，“total_count”:11｝
獲取集群信息
要顯示使用文檔，請運行磚集群得到——幫助．
數據庫集群獲取——cluster-id1234567890 - batch123
或者:
數據庫集群獲取——cluster-name my-cluster
｛“cluster_id”:“1234 - 567890 batch123”,“spark_context_id”:8232037838300762810,:“cluster_name my-cluster”,:“spark_version 8.1.x-scala2.12”," aws_attributes ": {:“zone_id us-west-2c”,“first_on_demand”:1、“可用性”:“SPOT_WITH_FALLBACK”,“spot_bid_price_percent”:100年,“ebs_volume_count”:0}，:“node_type_id i3.xlarge”,:“driver_node_type_id i3.xlarge”,“autotermination_minutes”:120年,“enable_elastic_disk”:假的," disk_spec ": {“disk_count”:0}，“cluster_source”:“用戶界麵”,“enable_local_disk_encryption”:假的," instance_source ": {:“node_type_id i3.xlarge”}，" driver_instance_source ": {:“node_type_id i3.xlarge”}，“狀態”:“終止”,"state_message": "非活動集群已終止(非活動120分鍾)."，“start_time”:1616773202562,“terminated_time”:1619228528317,“last_state_loss_time”:1619214150116,"自動定量":{“min_workers”:2“max_workers”:8}，" default_tags ": {“供應商”:“磚”,“創造者”:“someone@example.com”,:“ClusterName my-cluster”,“ClusterId”:“1234 - 567890 batch123”}，“creator_user_name”:“somone@example.com”," termination_reason ": {“代碼”:“不活躍”,“參數”:{“inactivity_duration_min”:“120”}，“類型”:“成功”}，“init_scripts_safe_mode”:假的｝
列出所有可用集群的信息
要顯示使用文檔，請運行磚集群列表——幫助．
數據庫集群列表——輸出JSON|金橋。
｛“集群”:(｛“cluster_id”:“1234 - 567890 batch123”,“spark_context_id”:8232037838300762810,:“cluster_name my-cluster”,:“spark_version 8.1.x-scala2.12”," aws_attributes ": {:“zone_id us-west-2c”,“first_on_demand”:1、“可用性”:“SPOT_WITH_FALLBACK”,“spot_bid_price_percent”:100年,“ebs_volume_count”:0}，:“node_type_id i3.xlarge”,:“driver_node_type_id i3.xlarge”,“autotermination_minutes”:120年,“enable_elastic_disk”:假的," disk_spec ": {“disk_count”:0}，“cluster_source”:“用戶界麵”,“enable_local_disk_encryption”:假的," instance_source ": {:“node_type_id i3.xlarge”}，" driver_instance_source ": {:“node_type_id i3.xlarge”}，“狀態”:“終止”,"state_message": "非活動集群已終止(非活動120分鍾)."，“start_time”:1616773202562,“terminated_time”:1619228528317,“last_state_loss_time”:1619214150116,"自動定量":{“min_workers”:2“max_workers”:8}，" default_tags ": {“供應商”:“磚”,“創造者”:“someone@example.com”,:“ClusterName my-cluster”,“ClusterId”:“1234 - 567890 batch123”}，“creator_user_name”:“somone@example.com”," termination_reason ": {“代碼”:“不活躍”,“參數”:{“inactivity_duration_min”:“120”}，“類型”:“成功”}，“init_scripts_safe_mode”:假的}，.．.］｝
列出可用的集群節點類型
要顯示使用文檔，請運行磚集群list-node-types——幫助．
數據庫集群列表-節點類型
｛“node_type”:(｛:“node_type_id z1d.12xlarge”,“memory_mb”:393216年,“num_cores”:48.0,“描述”:“z1d.12xlarge”,:“instance_type_id z1d.12xlarge”,“is_deprecated”:假的,“類別”:“內存優化”，“support_ebs_volumes”:沒錯,“support_cluster_tags”:沒錯,“num_gpus”:0," node_instance_type ": {:“instance_type_id z1d.12xlarge”,“local_disks”:2“local_disk_size_gb”:900年,"instance_family": "EC2 z1d Family vcpu "，“swap_size”:“10 g”}，“is_hidden”:假的,“support_port_forwarding”:沒錯,“display_order”:0,“is_io_cache_enabled”:假的}，.．.］｝
列出可用於創建集群的可用分區
要顯示使用文檔，請運行磚集群list-zones——幫助．
數據庫集群列表區域
｛“區域”:(“us-west-2c”,“us-west-2a”,“us-west-2b”),:“default_zone us-west-2c”｝