Skip to content
GitLab
菜单
为什么选择 GitLab
定价
联系销售
探索
为什么选择 GitLab
定价
联系销售
探索
登录
获取免费试用
主导航
搜索或转到…
项目
B
beam
管理
动态
成员
计划
Wiki
代码
仓库
分支
提交
标签
仓库图
比较修订版本
代码片段
锁定的文件
部署
发布
模型注册表
分析
贡献者分析
仓库分析
洞察
模型实验
效能分析
帮助
帮助
支持
GitLab 文档
比较 GitLab 各版本
社区论坛
为极狐GitLab 提交贡献
提交反馈
隐私声明
快捷键
?
新增功能
4
代码片段
群组
项目
显示更多面包屑
oss-mirrors
beam
提交
fbaf1f75
提交
fbaf1f75
编辑于
2 years ago
作者:
Chamikara Jayalath
浏览文件
操作
下载
补丁
差异文件
Updates Multi-language Java examples documentation
上级
2bf07953
No related branches found
No related tags found
无相关合并请求
变更
2
隐藏空白变更内容
行内
左右并排
显示
2 个更改的文件
examples/multi-language/README.md
+23
-2
23 个添加, 2 个删除
examples/multi-language/README.md
website/www/site/content/en/documentation/sdks/java-multi-language-pipelines.md
+4
-4
4 个添加, 4 个删除
...nt/en/documentation/sdks/java-multi-language-pipelines.md
有
27 个添加
和
6 个删除
examples/multi-language/README.md
+
23
−
2
浏览文件 @
fbaf1f75
...
...
@@ -126,9 +126,25 @@ gsutil cat gs://$GCP_BUCKET/multi-language-beam/output*
#### Instructions for running the Java pipeline at HEAD (Beam 2.41.0 and 2.42.0).
*
Activate a new virtual environment following
[
these instructions
](
https://beam.apache.org/get-started/quickstart-py/#create-and-activate-a-virtual-environment
)
.
*
2. Install Apache Beam package with gcp support and the
`sklearn`
package.
```
pip install apache-beam[gcp]
pip install sklearn
```
*
Startup the expansion service
```
python -m apache_beam.runners.portability.expansion_service_main -p <PORT> --fully_qualified_name_glob "*"
```
*
Make sure that Docker is installed and available on your system.
*
B
uild and push Python and Java Docker containers.
*
In a different shell, b
uild and push Python and Java Docker containers.
```
export DOCKER_ROOT=<Docker root>
...
...
@@ -137,7 +153,7 @@ export DOCKER_ROOT=<Docker root>
docker push $DOCKER_ROOT/beam_python3.8_sdk:latest
./gradlew :sdks:java:container:java11:docker -Pdocker-repository-root=$DOCKER_ROOT -Pdocker-tag=latest
./gradlew :sdks:java:container:java11:docker -Pdocker-repository-root=$DOCKER_ROOT -Pdocker-tag=latest
-Pjava11Home=$JAVA_HOME
docker push $DOCKER_ROOT/beam_java11_sdk:latest
```
...
...
@@ -149,6 +165,10 @@ Note that we override both the Java and Python SDK harness containers here.
export GCP_PROJECT=<GCP project>
export GCP_BUCKET=<GCP bucket>
export GCP_REGION=<GCP region>
export EXPANSION_SERVICE_PORT=<PORT>
# This removes any existing output.
gsutil rm gs://$GCP_BUCKET/multi-language-beam/output*
./gradlew :examples:multi-language:sklearnMinstClassification --args=" \
--runner=DataflowRunner \
...
...
@@ -157,6 +177,7 @@ export GCP_REGION=<GCP region>
--output=gs://$GCP_BUCKET/multi-language-beam/output \
--sdkContainerImage=$DOCKER_ROOT/beam_java11_sdk:latest \
--sdkHarnessContainerImageOverrides=.*python.*,$DOCKER_ROOT/beam_python3.8_sdk:latest \
--expansionService=localhost:$EXPANSION_SERVICE_PORT \
--region=${GCP_REGION}"
```
...
...
此差异已折叠。
点击以展开。
website/www/site/content/en/documentation/sdks/java-multi-language-pipelines.md
+
4
−
4
浏览文件 @
fbaf1f75
...
...
@@ -188,7 +188,7 @@ python -m apache_beam.runners.portability.local_job_service_main -p $JOB_SERVER_
(this guide requires that your JAVA_HOME is set to Java 11).
```
./gradlew :sdks:java:container:java11:docker
./gradlew :sdks:java:container:java11:docker
-Pjava11Home=$JAVA_HOME
```
5.
Run the pipeline.
...
...
@@ -243,9 +243,9 @@ pip install apache-beam[gcp,dataframe]
4.
Run the following command
```
python -m apache_beam.runners.portability.expansion_service_main -p <PORT> --fully_qualified_name_glob "*"
```
```
python -m apache_beam.runners.portability.expansion_service_main -p <PORT> --fully_qualified_name_glob "*"
```
The command runs
[
expansion_service_main.py
](
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/expansion_service_main.py
)
, which starts the standard expansion service. When you use
...
...
此差异已折叠。
点击以展开。
预览
0%
加载中
请重试
或
添加新附件
.
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
保存评论
取消
想要评论请
注册
或
登录