磚運行時為8.0毫升(不支持)
2021年3月磚公布的這張照片。
磚8.0運行時機器學習為機器學習和數據提供了一個準備好了環境科學的基礎上磚運行時的8.0(不支持)。磚運行時毫升含有許多流行的機器學習庫,包括TensorFlow PyTorch, XGBoost。它還支持使用Horovod分布深度學習培訓。
更多信息,包括指令創建磚集群運行時毫升、明白了介紹磚運行時機器學習。
請注意
這些版本注釋可能包括引用功能不可用這個版本的穀歌雲。
新功能和重大變化
磚磚的運行時8.0毫升之上8.0運行時。有什麼新信息的磚8.0運行時,包括Apache火花MLlib和SparkR,看到磚運行時的8.0(不支持)發行說明。
Conda通道配置
2020年9月蟒蛇公司更新了服務條款為anaconda.org頻道。基於新的服務條款你可能需要一個商業許可證如果你依靠蟒蛇的包裝和分布。訪問蟒蛇商業版常見問題解答為更多的信息。由於這一變化,我們刪除默認通道配置在磚Conda包管理器運行時8.0毫升。安裝或更新包使用% conda
命令,您必須指定一個通道。你使用任何蟒蛇的通道是由他們的服務條款。
重大變化磚Python運行時毫升的環境
看到磚運行時的8.0(不支持)對磚Python運行時環境的重大變化。安裝Python包的完整列表和他們的版本,看Python庫。
係統環境
磚的係統環境運行時8.0毫升不同於磚運行時的8.0如下:
DBUtils運行時毫升不包括:磚庫工具(dbutils.library)。使用
%皮普
和% conda
而不是命令。看到Notebook-scoped Python庫。
庫
下麵的章節列表庫包含在磚運行時的8.0毫升,不同於那些包含在磚8.0運行時。
Python庫
磚運行時8.0毫升使用Conda Python包管理,包括許多流行毫升包。
除了包中指定Conda環境在下麵幾節中,磚運行時8.0毫升的還包括以下方案:
hyperopt 0.2.5.db1
sparkdl 2.1.0.db4
Python庫對CPU集群
的名字:databricks-ml渠道:- - - - - -pytorch- - - - - -違約依賴關係:- - - - - -_libgcc_mutex = 0.1 =主要- - - - - -absl-py = 0.11.0 = pyhd3eb1b0_1它- - - - - -aiohttp = 3.6.3 = py38h7b6447c_0- - - - - -asn1crypto = 1.4.0 = py_0- - - - - -阿斯特= 0.8.1 = py38_0- - - - - -async-timeout = 3.0.1 = py38_0- - - - - -attrs = 20.3.0 = pyhd3eb1b0_0- - - - - -backcall = 0.2.0 = py_0- - - - - -bcrypt = 3.2.0 = py38h7b6447c_0- - - - - -布拉斯特區= 1.0 = mkl- - - - - -信號燈= 1.4 = py38_0- - - - - -boto3 = 1.16.7 = pyhd3eb1b0_0- - - - - -botocore = 1.19.7 = pyhd3eb1b0_0- - - - - -brotlipy = 0.7.0 = py38h27cfd23_1003- - - - - -c-ares = 1.17.1 = h27cfd23_0- - - - - -ca證書= 2021.4.13 = h06a4308_12021年5月26日#(更新2021.1.19維護更新)- - - - - -cachetools = 4.2.0 = pyhd3eb1b0_0- - - - - -certifi = 2020.12.5 = py38h06a4308_0- - - - - -cffi = 1.14.3 = py38h261ae71_2- - - - - -chardet = 3.0.4 = py38h06a4308_1003- - - - - -單擊= 7.1.2 = pyhd3eb1b0_0- - - - - -cloudpickle = 1.6.0 = py_0- - - - - -configparser = 5.0.1 = py_0- - - - - -cpuonly = 1.0 = 0- - - - - -密碼= 3.1.1 = py38h1ba5d50_0- - - - - -周期計= 0.10.0 = py38_0- - - - - -cython = 0.29.21 = py38h2531618_0- - - - - -decorator = 10/24/11 = py_0- - - - - -蒔蘿= 0.3.2 = py_0- - - - - -docutils = 0.15.2 = py38_0- - - - - -entrypoints = 0.3 = py38_0- - - - - -瓶= 1.1.2 = py_0- - - - - -freetype的= 2.10.4 = h5ab3b9f_0- - - - - -未來= 0.18.2 = py38_1- - - - - -gitdb =你= py_0- - - - - -gitpython = 3.1.11 = pyhd3eb1b0_1- - - - - -google-auth = 1.22.1 = py_0- - - - - -google-auth-oauthlib = 0.4.2 = pyhd3eb1b0_2- - - - - -google-pasta = 0.2.0 = py_0- - - - - -gunicorn = 20.0.4 = py38_0- - - - - -h5py = 2.10.0 = py38h7918eee_0- - - - - -hdf5 = 1.10.4 = hb1b8bf9_0- - - - - -icu = 58.2 = he6710b0_3- - - - - -idna = 2.10 = pyhd3eb1b0_0- - - - - -importlib-metadata = 2.0.0 = py_1- - - - - -intel-openmp = 2019.4 = 243- - - - - -ipykernel = 5.3.4 = py38h5ca1d4c_0- - - - - -ipython = 7.19.0 = py38hb070fc8_1- - - - - -ipython_genutils = 0.2.0 = pyhd3eb1b0_1- - - - - -isodate = 0.6.0 = py_1- - - - - -itsdangerous = 1.1.0 = py_0- - - - - -絕地= 0.17.2 = py38h06a4308_1- - - - - -jinja2 = 2.11.2 = pyhd3eb1b0_0- - - - - -jmespath = 0.10.0 = py_0- - - - - -joblib = 0.17.0 = py_0- - - - - -jpeg = 9 b = h024ee3a_2- - - - - -jupyter_client = 6.1.7 = py_0- - - - - -jupyter_core = 4.6.3 = py38_0- - - - - -= py38h2531618_0 1.3.0 kiwisolver =版本- - - - - -krb5 = 1.17.1 = h173b8e3_0- - - - - -lcms2 = 2.11 = h396b838_0- - - - - -ld_impl_linux - 64 = 2.33.1 = h53a641e_7- - - - - -libedit = 3.1.20191231 = h14c3975_1- - - - - -libffi = 3.3 = he6710b0_2- - - - - -libgcc-ng = 9.1.0 = hdf63c60_0- - - - - -libgfortran-ng = 7.3.0 = hdf63c60_0- - - - - -libpng = 1.6.37 = hbc83047_0- - - - - -libpq = 12.2 = h20c2e04_0- - - - - -libprotobuf = 3.13.0.1 = hd408876_0- - - - - -libsodium = 1.0.18 = h7b6447c_0- - - - - -libstdcxx-ng = 9.1.0 = hdf63c60_0- - - - - -libtiff = 4.1.0 = h2733197_1- - - - - -lightgbm = 3.1.1 = py38h2531618_0- - - - - -lz4-c = 1.9.2 = heb0550a_3- - - - - -尖吻鯖鯊= 1.1.3 = py_0- - - - - -減價= 3.3.2 = py38_0- - - - - -markupsafe = 1.1.1 = py38h7b6447c_0- - - - - -matplotlib-base = 3.2.2 = py38hef1b27d_0- - - - - -mkl = 2019.4 = 243- - - - - -mkl-service = tripwire = py38he904b0f_0- - - - - -mkl_fft = 1.2.0 = py38h23d657b_0- - - - - -mkl_random = 1.1.0 = py38h962f231_0- - - - - -more-itertools = 8.6.0 = pyhd3eb1b0_0- - - - - -multidict = 4.7.6 = py38h7b6447c_1- - - - - -ncurses = 6.2 = he6710b0_1- - - - - -networkx = 2.5 = py_0- - - - - -nltk = 3.5 = py_0- - - - - -numpy = 1.19.2 = py38h54aff64_0- - - - - -numpy-base = 1.19.2 = py38hfa32c7d_0- - - - - -oauthlib = 3.1.0 = py_0- - - - - -olefile = 0.46 = py_0- - - - - -openssl = 1.1.1k = h27cfd23_02021年5月26日#(更新1.1.1i維護更新)- - - - - -包裝= 20.4 = py_0- - - - - -熊貓= 1.1.3 = py38he6710b0_0- - - - - -paramiko = 2.7.2 = py_0- - - - - -parso = 0.7.0 = py_0- - - - - -容易受騙的人= 0.5.1 = py38_0- - - - - -pexpect = 4.8.0 = pyhd3eb1b0_3- - - - - -pickleshare = 0.7.5 = pyhd3eb1b0_1003- - - - - -枕頭= 8.0.1 = py38he98fc37_0- - - - - -皮普= 20.2.4 = py38h06a4308_0- - - - - -情節= 4.14.2 = pyhd3eb1b0_0- - - - - -prompt-toolkit = 3.0.8 = py_0- - - - - -prompt_toolkit = 3.0.8 = 0- - - - - -protobuf = 3.13.0.1 = py38he6710b0_1- - - - - -psutil = 5.7.2 = py38h7b6447c_0- - - - - -psycopg2 = 2.8.5 = py38h3c74f83_1- - - - - -ptyprocess = 0.6.0 = pyhd3eb1b0_2- - - - - -pyasn1 = 0.4.8 = py_0- - - - - -pyasn1-modules = 0.2.8 = py_0- - - - - -pycparser = 2.20 = py_2- - - - - -pygments = 2.7.2 = pyhd3eb1b0_0- - - - - -pyjwt = 1.7.1上= py38_0- - - - - -pynacl = 1.4.0 = py38h7b6447c_1- - - - - -pyodbc = 4.0.30 = py38he6710b0_0- - - - - -pyopenssl = 19.1.0 = pyhd3eb1b0_1- - - - - -pyparsing = 2.4.7 = pyhd3eb1b0_0- - - - - -pysocks = 1.7.1上= py38h06a4308_0- - - - - -python = 3.8.8 = hdb3f193_42021年5月26日#(更新3.8.5維護更新)- - - - - -python-dateutil = 2.8.1發布= py_0- - - - - -python編輯器的1.0.4 = = py_0- - - - - -pytz = 2020.1 = py_0- - - - - -pyzmq = 19.0.2 = py38he6710b0_1- - - - - -readline = 8.0 = h7b6447c_0- - - - - -regex = 2020.10.15 = py38h7b6447c_0- - - - - -= 2.24.0 = py_0請求- - - - - -= py_0 1.3.0 requests-oauthlib =版本- - - - - -重試= 1.3.3 = py_2- - - - - -rsa = 4.7 = pyhd3eb1b0_1- - - - - -s3transfer = 0.3.4 = pyhd3eb1b0_0- - - - - -scikit-learn = 0.23.2 = py38h0573a6f_0- - - - - -scipy = 1.5.2 = py38h0b6359f_0- - - - - -setuptools = 50.3.1 = py38h06a4308_1- - - - - -simplejson = 3.17.2 = py38h7b6447c_0- - - - - -6 = 1.15.0 = py38h06a4308_0- - - - - -smmap = 3.0.4 = py_0- - - - - -sqlite = 3.33.0 = h62c20be_0- - - - - -sqlparse = 0.4.1 = py_0- - - - - -statsmodels = 0.12.0 = py38h7b6447c_0- - - - - -彙總= 0.8.7 = py38_0- - - - - -threadpoolctl = 2.1.0 = pyh5ca1d4c_0- - - - - -tk = 8.6.10 = hbc83047_0- - - - - -龍卷風= 6.0.4 = py38h7b6447c_1- - - - - -tqdm = 4.50.2 = py_0- - - - - -traitlets = 5.0.5 = py_0- - - - - -typing_extensions = 3.7.4.3 = py_0- - - - - -unixodbc = 2.3.9 = h7b6447c_0- - - - - -urllib3 = 1.25.11 = py_0- - - - - -wcwidth = 0.2.5 = py_0- - - - - -websocket-client = 0.57.0 = py38_2- - - - - -werkzeug = 1.0.1 = py_0- - - - - -輪= 0.35.1 = pyhd3eb1b0_0- - - - - -打包= 1.12.1 = py38h7b6447c_1- - - - - -xz = 5.2.5 = h7b6447c_0- - - - - -yarl = 1.6.3 = py38h27cfd23_0- - - - - -zeromq = 4.3.3 = he6710b0_3- - - - - -氧化鋅碘仿糊= 3.4.0 = pyhd3eb1b0_0- - - - - -zlib = 1.2.11 = h7b6447c_3- - - - - -zstd = 1.4.5 = h9ceee32_0- - - - - -皮普:- - - - - -astunparse = = 1.6.3- - - - - -azure-core = = 1.10.0- - - - - -azure-storage-blob = = 12.7.0- - - - - -databricks-cli = = 0.14.1- - - - - -diskcache = = 5.1.0- - - - - -4.4.1碼頭工人= =- - - - - -flatbuffers = = 1.12- - - - - -恐嚇= = 0.3.3- - - - - -grpcio = = 1.32.0- - - - - -horovod = = 0.21.1- - - - - -joblibspark = = 0.3.0- - - - - -keras-preprocessing = = 1.1.2- - - - - -考拉= = 1.5.0- - - - - -llvmlite = = 0.35.0- - - - - -mleap = = 0.16.1- - - - - -mlflow = = 1.13.1- - - - - -msr = = 0.6.19- - - - - -numba = = 0.52.0- - - - - -opt-einsum = = 3.3.0- - - - - -petastorm = = 0.9.8- - - - - -pyarrow = = 1.0.1- - - - - -pyyaml = = 5.4- - - - - -querystring-parser = = 4- - - - - -seaborn = = 0.10.0- - - - - -世鵬科技電子= = 0.37.0- - - - - -切片機= = 0.0.3- - - - - -spark-tensorflow-distributor = = 0.1.0- - - - - -tensorboard = = 2.4.1- - - - - -tensorboard-plugin-wit = = 1.8.0- - - - - -tensorflow-cpu = =測試盒框- - - - - -tensorflow-estimator = =測試盒框- - - - - -termcolor = = 1.1.0- - - - - -火炬= = 1.7.1上- - - - - -torchvision = = 0.8.2- - - - - -xgboost = = 1.3.1前綴:/磚/ conda / env / databricks-ml
Java和Scala庫(Scala 2.12集群)
除了Java和Scala庫磚8.0運行時,磚運行時8.0毫升包含以下jar:
CPU集群
組ID |
工件ID |
版本 |
---|---|---|
com.typesafe.akka |
akka-actor_2.12 |
2.5.23 |
ml.combust.mleap |
mleap-databricks-runtime_2.12 |
0.17.0-4882dc3 |
ml.dmlc |
xgboost4j-spark_2.12 |
1.3.1 |
ml.dmlc |
xgboost4j_2.12 |
1.3.1 |
org.graphframes |
graphframes_2.12 |
0.8.1-db2-spark3.1 |
org.mlflow |
mlflow-client |
1.13.1 |
org.mlflow |
mlflow-spark |
1.13.1 |
org.scala-lang.modules |
scala-java8-compat_2.12 |
0.8.0 |
org.tensorflow |
spark-tensorflow-connector_2.12 |
1.15.0 |