← 开源
flink-extended

dl-on-flink

Deep Learning on Flink aims to integrate Flink and deep learning frameworks (e.g. TensorFlow, PyTorch, etc) to enable distributed deep learning training and inference on a Flink cluster.

Model DevelopmentDeep Learning FrameworksJava
在 GitHub 打开
增长势头
+024 小时新增 Star0.0%
694
Star
196
Fork
+0
本周
17
贡献者
创建于 2019-06-17 · 更新于 2026-09-26 · 今日第 14901 名
主要开发者

flink-extended/dl-on-flink 排名第 14,901,有 694 个星标。

同类别比较

flink-extended/dl-on-flink 排名第 14,901,此处有 8 个同类条目;khalil-research/PyEPO 排名第 14,902。

同类别比较
名称排名指标
HasnainRaz/Fast-SRGAN14,879
语言:
Python
星标数:
695
分叉数:
120
简介:
A Fast Deep Learning Model to Upsample Low Resolution Videos to High Resolution at 30fps
aleju/mario-ai14,883
语言:
Lua
星标数:
695
分叉数:
142
简介:
Playing Mario with Deep Reinforcement Learning
mv-lab/swin2sr14,889
语言:
Python
星标数:
695
分叉数:
83
简介:
[ECCV] Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration. Advances in Image Manipulation (AIM) workshop ECCV 2022. Try it out! over 3.3M runs https://replicate.com/mv-lab/swin2sr
soobinseo/Transformer-TTS14,890
语言:
Python
星标数:
695
分叉数:
139
简介:
A Pytorch Implementation of "Neural Speech Synthesis with Transformer Network"
khalil-research/PyEPO14,902
语言:
Jupyter Notebook
星标数:
694
分叉数:
90
简介:
A PyTorch-based End-to-End Predict-then-Optimize Library for Linear and Integer Programming
KarhouTam/FL-bench14,908
语言:
Python
星标数:
693
分叉数:
116
简介:
Benchmark of federated learning. Dedicated to the community. 🤗
ankane/eps14,913
语言:
Ruby
星标数:
693
分叉数:
15
简介:
Machine learning for Ruby
elastic/eland14,915
语言:
Python
星标数:
693
分叉数:
112
简介:
Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
关键参数与规格
关键参数与规格
参数值
语言Java
类别Model Development
子类别Deep Learning Frameworks
关键词Distributed training
星标数694
分叉数196
贡献者17
创建日期2019-06-17
更新日期2026-09-26
1 天星标数0
7 天星标数0
排名14,901
上次排名14,736
排名变化-165
7 天星标排名14,709
上次 7 天星标排名14,718
7 天星标排名变化9
总星标排名14,978
上次总星标排名14,955
总星标排名变化-23
README 摘录

Deep Learning on Flink Deep Learning on Flink aims to integrate Flink and deep learning frameworks (e. ) to enable distributed deep learning training and inference on a Flink cluster. It runs the deep learning tasks inside a Flink operator so that Flink can help establish a distributed environment, manage the resource, read/write the data with the rich connectors in Flink and handle the failures. Currently, Deep Learning on Flink supports TensorFlow. Supported Operating System Deep Learning on Flink is

tested and supported on the following 64-bit systems: Ubuntu 18. 15 Support Framework Version TensorFlow: 1. x Getting Started Deep learning on Flink currently works with Tensorflow and PyTorch. You can see the following pages for the usage and examples. DL on Flink with Tensorflow DL on Flink with PyTorch Build From Source **Requirements** python: 3. 0 Deep Learning on Flink requires Java and Python works together. Thus, we need to build for both Java and Python. Initializing Submodules

before Building Deep Learning on Flink from Source Please use the following command to initialize submodules before building from source. Build Java After finish, you can find the target distribution in the folder. Build Python Install from Source You can run the following commands to install the Python packages from source Build wheels We provide a script to build wheels for Python packages, you can run the following command. After finish, you can find the wheels at . Then you can

install the python package with the wheels. For More Information Design document License Apache License 2.

常见问题

记录了多少星标和分叉?

flink-extended/dl-on-flink 有 694 个星标和 196 个分叉。

仓库排名是多少?

flink-extended/dl-on-flink 排名第 14,901。

记录的语言和类别是什么?

flink-extended/dl-on-flink 有 694 个星标,语言为 Java,类别为 Model Development。

哪个仓库排名最接近?

flink-extended/dl-on-flink 与 khalil-research/PyEPO 比较,后者排名第 14,902,有 694 个星标。

README

Deep Learning on Flink

Deep Learning on Flink aims to integrate Flink and deep learning frameworks (e.g. TensorFlow, PyTorch, etc.) to enable distributed deep learning training and inference on a Flink cluster.

It runs the deep learning tasks inside a Flink operator so that Flink can help establish a distributed environment, manage the resource, read/write the data with the rich connectors in Flink and handle the failures.

Currently, Deep Learning on Flink supports TensorFlow.

Supported Operating System

Deep Learning on Flink is tested and supported on the following 64-bit systems:

  • Ubuntu 18.04
  • macOS 10.15

Support Framework Version

  • TensorFlow: 1.15.x & 2.4.x
  • PyTorch: 1.11.x
  • Flink: 1.14.x

Getting Started

Deep learning on Flink currently works with Tensorflow and PyTorch. You can see the following pages for the usage and examples.

Build From Source

Requirements

  • python: 3.7
  • cmake >= 3.6
  • java 1.8
  • maven >=3.3.0

Deep Learning on Flink requires Java and Python works together. Thus, we need to build for both Java and Python.

Initializing Submodules before Building Deep Learning on Flink from Source

Please use the following command to initialize submodules before building from source.

git submodule update --init --recursive

Build Java

mvn -DskipTests clean install

After finish, you can find the target distribution in the dl-on-flink-dist/target folder.

Build Python

Install from Source

You can run the following commands to install the Python packages from source

# Install dl-on-flink-framework first
pip install dl-on-flink-framework/python

# Note that you should only install one of the following as they require
# different versions of Tensorflow 
# For tensorflow 1.15.x
pip install dl-on-flink-tensorflow/python
# For tensorflow 2.4.x
pip install dl-on-flink-tensorflow-2.x/python

Build wheels

We provide a script to build wheels for Python packages, you can run the following command.

bash tools/build_wheel.sh

After finish, you can find the wheels at tools/dist. Then you can install the python package with the wheels.

pip install tools/dist/

For More Information

Design document

License

Apache License 2.0

排名相邻
同类项目