Skip to content
代码片段 群组 项目
未验证 提交 95be6b9b 编辑于 作者: Bruno Volpato's avatar Bruno Volpato 提交者: GitHub
浏览文件

[JdbcIO] Add fetchSize to the schema provider for partitioned reads (#29015)

上级 4f4a58a2
No related branches found
No related tags found
无相关合并请求
......@@ -1288,6 +1288,12 @@ public class JdbcIO {
return toBuilder().setPartitionColumn(partitionColumn).build();
}
/** The number of rows to fetch from the database in the same {@link ResultSet} round-trip. */
public ReadWithPartitions<T, PartitionColumnT> withFetchSize(int fetchSize) {
checkArgument(fetchSize > 0, "fetchSize can not be less than 1");
return toBuilder().setFetchSize(fetchSize).build();
}
/** Data output type is {@link Row}, and schema is auto-inferred from the database. */
public ReadWithPartitions<T, PartitionColumnT> withRowOutput() {
return toBuilder().setUseBeamSchema(true).build();
......
......@@ -134,6 +134,12 @@ public class JdbcSchemaIOProvider implements SchemaIOProvider {
if (partitions != null) {
readRows = readRows.withNumPartitions(partitions);
}
@Nullable Short fetchSize = config.getInt16("fetchSize");
if (fetchSize != null) {
readRows = readRows.withFetchSize(fetchSize);
}
return input.apply(readRows);
} else {
......
0% 加载中 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册