python分布式执行框架Ray的介绍

说明

1、Ray为构建分布式应用程序提供了简单通用的API。

2、Ray是一种便于大规模应用和先进机器学习库的分布式执行框架。

Ray通过以下方式完成了这项任务：

为构建和运行分布式应用程序提供简单的原语。

最终用户可以并行化单个机器代码，代码变化很少到零。

为了支持复杂的应用，在核心Ray上包含了大量的应用程序、库和工具。

安装方法

[root@node2~]#pipinstall'ray[default]'
Lookinginindexes:https://mirrors.aliyun.com/pypi/simple/
Collectingray[default]
Downloadinghttps://mirrors.aliyun.com/pypi/packages/13/ec/f727ddd3fbcdc6102eace62cd9d9d9d9d9ad812d40eb7de876aca24dray-1.4.1-cp36-cp36m-manylinux2014_x86_64.whl(51.6MB)
|████████████████████████████████|51.6MB269kB/s
Collectingaiohttp
Downloadinghttps://mirrors.aliyun.com/pypi/packages/31/28/7b49246d7825c61a14306ab05ecf5b009b19cf6ea41267aiohttp-3.7.4.post0-cp36-cp36m-manylinux2014_x86_64.whl(1.3MB)
|████████████████████████████████|1.3MB121.5MB/s
Requirementalreadysatisfied:dataclassesin/usr/local/lib/python3.6/site-packages(fromray[default])(0.8)
Collectingprotobuf>=3.15.3
Downloadinghttps://mirrors.aliyun.com/pypi/packages/53/4e/e2db88d0bda6a79ea62fdbafdbaf813770ce3770d45bc580512d95protobuf-3.17.3-cp36-cp36m3-manylinux_2_5_x86_64.nylinux1.whl(1.0MB)
|████████████████████████████████|1.0MB136.1MB/s
Collectingjsonschema
Downloadinghttps://mirrors.aliyun.com/pypi/packages/c5/8f/51e89ce52a0854359217cdbf6e75e5d5b1d4b5ade40ce018b8jsonschema-3.2.0-py2.py3-none-any.whl(56kB)
|████████████████████████████████|56kB7.1MB/s
Collectingaioredis
Downloadinghttps://mirrors.aliyun.com/pypi/packages/b0/64/1b1612d0a104f21f80eb4c6e1b6075f2e6ababa22f46f229cfd3fdac859aioredis-1.3.1-py3-none-any.whl(65kB)
|████████████████████████████████|65kB6.3MB/s
Collectingpydantic>=1.8
Downloadinghttps://mirrors.aliyun.com/pypi/packages/2b/7c/7d0b3f2d7959b71930186db236deded165f9bca1b75f46f4cfa6fa6f4f9dpydantic-1.8.2-cp36-cp36m-manylinux2014_x86_64.whl(10.2MB)
|████████████████████████████████|10.2MB420kB/s
Collectingpyyaml
Downloadinghttps://mirrors.aliyun.com/pypi/packages/7a/5b/bc0bab38247ba158504a41012b6c03f153c652734ecece1849518PyYAML-5.4.1-cp36-cp36m-nylinux1.whl(640kB)
|████████████████████████████████|640kB19.1MB/s
Collectingnumpy>=1.16
Downloadinghttps://mirrors.aliyun.com/pypi/packages/14/32/d3fa649ad7ec0b8277b92fefd3c4d376b0b2373071546f38fa08numpy-19.5-cp36-cp36m-manylinux2010_x86_640.whl(14.8MB)
|████████████████████████████████|14.8MB296kB/s
Collectinggpustat
Downloadinghttps://mirrors.aliyun.com/pypi/packages/b4/69/d8c84971517aeabd61af7da080fdc60948ba396d242f472e43d008gpustat-0.6.0.tar.gz(78kB)
|████████████████████████████████|78kB7.3MB/s
Collectingredis>=3.5.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/a7/7c/24fb051df653cf1a5d938d8f5d19802a8cef25706fda242ff97b7redis-3.5.3-py2.py3-none-any.whl(72kB)
|████████████████████████████████|72kB968kB/s
Collectingaiohttp-cors
Downloadinghttps://mirrors.aliyun.com/pypi/packages13/e7/e436a0c0eb5127d8b491a9b83ecd2391c6ff7d5548dfaec2080a2340fd/e7/e7/e4361c0eb5127d8daiohttp_cors-0.7.0-py3-none-any.whl(27kB)
Collectingpy-spy>=0.2.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages9d/4d/1a9cbe9a0b543e673cb38cb38afe26451522a9ef8e4897b574cb76838f245py_spy-0.3.7-py2.py3-none-nylinux1.whl(3.1MB)
|████████████████████████████████|3.1MB131kB/s
Requirementalreadysatisfied:requestsin/usr/local/lib/python3.6/site-packages(fromray[default])(2.24.0)
Collectingcolorama
Downloadinghttps://mirrors.aliyun.com/pypi/packages/44/98/5b8627fbf250d239ae0ecb724f872af1c91f4a1edf4d36a2064colorama-0.4.4-py2.py3-none-any.whl(16kB)
Collectingfilelock
Downloadinghttps://mirrors.aliyun.com/pypi/packages/93/83/71a2e615b9f390c0dea167f81def86e18e1971b1ab01a35afilelock-3.0.12-py3-none-any.whl(7.6kB)
Collectingprometheus-client>=0.7.1
Downloadinghttps://mirrors.aliyun.com/pypi/packages/09/da/4e8471ff8257959581593b5b84769d32f58e573b59fccaf355d3529ad53prometheus_client-0.11.0-py2.py3-none-any.whl(56kB)
|████████████████████████████████|56kB4.2MB/s
Collectingmsgpack<2.0.0,>=1.0.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/0c/0d/b1d9d32d03ce38ba5e2a37fbae850afd4530cc41e8335f1865amsgpack-1.0.cp36-cp36m-manylinux1_x86.whl(272kB)
|████████████████████████████████|272kB127.6MB/s
Requirementalreadysatisfied:click>=7.0in/usr/local/lib/python3.6/site-packages(fromray[default])(7.1.2)
Collectingopencensus
Downloadinghttps://mirrors.aliyun.com/pypi/packages18/59/120441233d00f705383ad98579aeb0d82d66b3a254a21b54bb/120412041233d00f/1203383adopencensus-0.7.13-py2.py3-none-any.whl(127kB)
|████████████████████████████████|127kB128.9MB/s
Collectinggrpcio>=1.28.1
Downloadinghttps://mirrors.aliyun.com/pypi/packages/eb/32/32a2b0ce0d4040331952cef3d040d119a15b3c62b60bdcebe768bdedgrpcio-1.39.0-cp36-cp36m-manylinux2014_x86_64.whl(4.3MB)

|████████████████████████████████|4.3MB417kB/s
Collectingcolorful
Downloadinghttps://mirrors.aliyun.com/pypi/packages/b0/8e/e386e2462662d24d734c2f513f34d957032618c80e605dfaf6colorful-0.5.4-py2.py3-none-any.whl(201kB)
|████████████████████████████████|201kB113.9MB/s
Requirementalreadysatisfied:six>=1.5.2in/usr/local/lib/python3.6/site-packages(fromgrpcio>=1.28.1->ray[default])(1.15.0)
Requirementalreadysatisfied:typing-extensions>=3.7.4.3in/usr/local/lib/python3.6/site-packages(frompydantic>=1.8->ray[default])(3.10.0.0)
Collectingmultidict<7.0,>=4.5
Downloadinghttps://mirrors.aliyun.com/pypi/packages/a1/35/b22524d6bcfb4c5eff413a06bc6ea62ea654da5c6c9999ced5f/multidict-5.1.cp36-cp36m-manylinux2014_x86.whl(141kB)
|████████████████████████████████|141kB120.6MB/s
Collectingidna-ssl>=1.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/46/03/07c4894ae38b0de52b5256b24bf189bb83e4dabfe2c8f24eec6f4idna-ssl-1.1.0.tar.gz(3.4kB)
Collectingattrs>=17.3.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/20/a9/ba6f1cd1a1517ff02b35acd6a7e426371dfab08e42b29b6d0713c/attrs-21.2.0-py2.py3-none-any.whl(53kB)
|████████████████████████████████|53kB3.9MB/s
Requirementalreadysatisfied:chardet<5.0,>=2.0in/usr/local/lib/python3.6/site-packages(fromaiohttp->ray[default])(3.0.4)
Collectingyarl<2.0,>=1.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/da/08/52b26b4bce7b818b410ae37c5e424c9ea420ca97dadac29b15yarl-1.6.3-cp36-cp36m-manylinux2014_x86_64.whl(293kB)
|████████████████████████████████|293kB130.8MB/s
Collectingasync-timeout<4.0,>=3.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/e1/1e/5a441be210726c464f33c8b196272f60675a9d2e46c18e65ec4async_timeout-3.0.1-py3-none-any.whl(8.2kB)
Requirementalreadysatisfied:idna>=2.0in/usr/local/lib/python3.6/site-packages(fromidna-ssl>=1.0->aiohttp->ray[default])(2.10)
Collectinghiredis
Downloadinghttps://mirrors.aliyun.com/pypi/packages/82/72/313fb6d30b713c86247f9abe91054ceb926ecb08d44afbhiredis-2.0.0-cp36-cp36m-manylinux2010.whl(84kB)
|████████████████████████████████|84kB4.8MB/s
Collectingnvidia-ml-py3>=7.352.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages6d/64/cce82bddb80c0b0f5c703bbdafa94bfb69a1c5ad79cf00b482468f0d3nvidia-ml-py3-7.352.0.tar.gz(19kB)
Collectingpsutil
Downloadinghttps://mirrors.aliyun.com/pypi/packages/da/82/56cd16a4c5f53e5d7b2c30d5c803e124f218eb64ca9c30bc907ead/psutil-5.8.0-cp36-cp36m-manylinux2010.whl(291kB)
|████████████████████████████████|291kB131.6MB/s
Collectingblessings>=1.6
Downloadinghttps://mirrors.aliyun.com/pypi/packages/03/74/489f85a78247609c6b4f1373cbf3ba0d864b1a56617b645d6fd2a4ablessings-1.7-py3-none-any.whl(18kB)
Requirementalreadysatisfied:setuptoolsin/usr/lib/python3.6/site-packages(fromjsonschema->ray[default])(39.2.0)
Collectingpyrsistent>=0.14.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/6c/19/1af501f6f38a40ede6d0185ba481bdb18ffc9deab0d0d0db173bc0f4pyrsistent-0.18.0-cp36-cp36m-manylinux1_x86_64.whl(117kB)
|████████████████████████████████|117kB138.2MB/s
Requirementalreadysatisfied:importlib-metadatain/usr/local/lib/python3.6/site-packages(fromjsonschema->ray[default])(4.3.0)
Requirementalreadysatisfied:zipp>=0.5in/usr/local/lib/python3.6/site-packages(fromimportlib-metadata->jsonschema->ray[default])(3.4.1)
Collectingopencensus-context==0.1.2
Downloadinghttps://mirrors.aliyun.com/pypi/packages/f1/33/990f1be770fc8d3c154fc24743a96f16a0e14e1b7540c2fd93opencensus_context-0.1.2-py2.py3-none-any.whl(4.4kB)
Collectinggoogle-api-core<2.0.0,>=1.0.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/aa/51/629a31d4db2bc0b789ac4034108c647bea9cf5b54c866df47begoogle_api_core-1.31.0-py2.py3-none-any.whl(93kB)
|████████████████████████████████|93kB1.6MB/s
Collectingcontextvars
Downloadinghttps://mirrors.aliyun.com/pypi/packages/83/96/55b82d9f1373be9d67262e1b8106cb83ed7c2fa5bcb7cd97e9contextvars-2.4.tar.gz(9.6kB)
Collectinggoogle-auth<2.0dev,>=1.25.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/d7/13/35b1e0a63e160ecad04985e60339382e60222f8228ab226c9google_auth-1.33.1-py2.py3-none-any.whl(152kB)
|████████████████████████████████|152kB127.7MB/s
Collectingpackaging>=14.3
Downloadinghttps://mirrors.aliyun.com/pypi/packages/3c/77/e262b66dc5008dbe423070d957fa03be5da2ba110581900fda546packaging-21.0-py3-none-any.whl(40kB)
|████████████████████████████████|40kB6.1MB/s
Collectinggoogleapis-common-protos<2.0dev,>=1.6.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/55/08/796a6bc0b50e2b71601c953d3d5100016abea106131d71e56e7bgoogleapis_common_protos-1.53.0-py2.py3-none-any.whl(198kB)
|████████████████████████████████|198kB136.3MB/s
Collectingsetuptools
Downloadinghttps://mirrors.aliyun.com/pypi/packages/bd/25/5bdf71adebe3fa76b2e2f045ae53e208e40a4231ad00c300e4353setuptools-57.4.0-py3-none-any.whl(819kB)
|████████████████████████████████|819kB127.0MB/s
Requirementalreadysatisfied:pytzin/usr/local/lib/python3.6/site-packages(fromgoogle-api-core<2.0.0,>=1.0.0->opencensus->ray[default])(2020.1)
Collectingcachetools<5.0,>=2.0.0
Downloadinghttps://mirrors.aliyun.com/pypi/packages/bf/28/c4f576c67ad06b91d98d543a5e01805cf065e00871f7e52d2a31adcachetools-4.2.2-py3-none-any.whl(11kB)
Collectingpyasn1-modules>=0.2.1
Downloadinghttps://mirrors.aliyun.com/pypi/packages/95/de/214830a982a3e286c3794f4ae67a495df1108c3da8a9f62159b9a9d/pyasn1modules-0.2.8-py2.py3-none-any.whl(155kB)
|████████████████████████████████|155kB21.5MB/s
Collectingrsa<5,>=3.1.4
Downloadinghttps://mirrors.aliyun.com/pypi/packages/e9/93/0c0f002031f18b53af7a6166103c02b9c0667be528944137c954ec921b3//rsa-4.7.2-py3-none-any.whl(34kB)
Collectingpyparsing>=2.0.2
Downloadinghttps://mirrors.aliyun.com/pypi/packages/8a/bb/48841f56197b13700afd5658fc279a2025a39e249b7cf296669b15dpyparsing-2.4.7-py2.py3-none-any.whl(67kB)
|████████████████████████████████|67kB7.2MB/s
collectingpyasn<0.5.0,>=0.4.6
Downloadinghttps://mirrors.aliyun.com/pypi/packages/62/1e/a94a8d635fa3ce4cfcfc7f506003548d0a247ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl(77kB)
|████████████████████████████████|77kB7.0MB/s
Requirementalreadysatisfied:certifi>=2017.4.17in/usr/local/lib/python3.6/site-packages(fromrequests->ray[default])(2020.6.20)
Requirementalreadysatisfied:urllib3！=1.25.0,!=1.25.1,<1.26,>=1.21.1in/usr/local/lib/python3.6/site-packages(fromrequests->ray[default])(1.25.10)
Collectingimmutables>=0.9
Downloadinghttps://mirrors.aliyun.com/pypi/packages/4a/52/e64a14a99cbfe0405e9f076aef031cb954a3efa1d5bacd524978aimmutables-0.15-cp36-cp36m-manylinux1_x86.whl(100kB)
|████████████████████████████████|100kB18.2MB/s
Usinglegacy'setup.pyinstall'foridna-ssl,sincepackage'wheel'isnotinstalled.
Usinglegacy'setup.pyinstall'forgpustat,sincepackage'wheel'isnotinstalled.
Usinglegacy'setup.pyinstall'fornvidia-ml-py3,sincepackage'wheel'isnotinstalled.
Usinglegacy'setup.pyinstall'forcontextvars,sincepackage'wheel'isnotinstalled.
Installingcollectedpackages:pyasn1，setuptools,rsa,pyparsing,pyasn1-modules,protobuf,multidict,immutables,cachetools,yarl,packaging,idna-ssl,googleapis-common-protos,google-auth,contextvars,attrs,async-timeout,pyrsistent,psutil,opencensus-context,nvidia-ml-py3,hiredis,google-api-core,blessings,aiohttp,redis,pyyaml,pydantic,py-spy,prometheus-client,opencensus,numpy,msgpack,jsonschema,grpcio,gpustat,filelock,colorama,aioredis,aiohttp-cors,ray,colorful
Attemptinguninstall:setuptools
Foundexistinginstallation:setuptols39.2.0
Uninstallingsetuptools-39.2.0:
Successfullyuninstalledsetuptools-39.2.0
Runningsetup.pyinstallforidna-ssl...done
Runningsetup.pyinstallforcontextvars...done
Runningsetup.pyinstallfornvidia-ml-py3...done
Runningsetup.pyinstallforgpustat...done
Successfullyinstalledaiohttp-3.7.4.post0aiohttp-cors-0.7.aioredis-1.3.1async-timeout-3.0.1atttrs-21.2.blessings-1.7cachetools-4.2.colorama-0.4.4colorful-0.5.4contextvars-2.4filelock-3.0.12gogle-api-core-1.31.0google-auth-1.33.1googleapis-common-protos-1.53.0gpustat-0.6.1.39.0grpciohiredis-2.0.0idna-ssl-1.1.0immutables-0.15jsonschema-3.2.0msgpack-1.00.2multidict-.0numpy-1.19.5nvidia-ml-py3-7.352.0opencensus-0.7.13opencensus-context-0.1.2packaging-21.prometheus-client-0.11.0protobuf-3.17.3psutil-5.8.0py-spy-0.3.7pyasn1-0.4.8pyasn1-modules-0.2.8pydantic-1.8.2pyparsing-2.4.7pyrstent-0.18.pyyaml-5.4.1ray-1.4.1redis-3.5.5..3rsa-4.7.setuptools-57.4.0yarl-1.6.3

以上是python分布式执行框架Ray的介绍，希望对大家有所帮助。更多Python学习指导：基础教程python基础教程

本文教程操作环境：windows7系统Python 3.9.1，DELL G3电脑。