[BEAM-9722] added SnowflakeIO with Read operation (#11360)
* [BEAM-9722] added SnowflakeIO with Read operation * [BEAM-9722] Added SnowflakeCloudProvider to enable use various clouds with Snowflake * [BEAM-9722] added docstrings for public methods * [BEAM-9722] Added changed cleanup staged GCS files to Beam FileSystems * [BEAM-9722] Added javadocs for public methods in DataSourceConfiguration * add testing p8 file to RAT exclude refactor SnowflakeCredentials add information about possibly left files on cloud storage small docs changes * documentation changes * [BEAM-9722] Added TestRule and changed Unit tests to use pipeline.run * [BEAM-9722] Renamed Snowflake Read unit test and applied spotless * [BEAM-9722] remove SnowflakeCloudProvider interface * [BEAM-9722] doc changes * [BEAM-9722] add `withoutValidation` to disable verifying connection to Snowflake during pipeline construction * [BEAM-9722] added MoveOption and removed leftover file * [BEAM-9722] fixed tests. Add tests for `withQuery` * [BEAM-9722] make `CopyIntoStageFn` retryable * [BEAM-9722] added `Reshuffle` step after `CopyIntoStageFn` Co-authored-by:Kasia Kucharczyk <katarzyna.kucharczyk@polidea.com> Co-authored-by:
pawel.urbanowicz <pawel.urbanowicz@polidea.com>
显示
- CHANGES.md 1 个添加, 0 个删除CHANGES.md
- build.gradle 3 个添加, 0 个删除build.gradle
- sdks/java/io/snowflake/build.gradle 42 个添加, 0 个删除sdks/java/io/snowflake/build.gradle
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/CloudProvider.java 32 个添加, 0 个删除.../java/org/apache/beam/sdk/io/snowflake/CloudProvider.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeIO.java 759 个添加, 0 个删除...in/java/org/apache/beam/sdk/io/snowflake/SnowflakeIO.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakePipelineOptions.java 133 个添加, 0 个删除...pache/beam/sdk/io/snowflake/SnowflakePipelineOptions.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeService.java 36 个添加, 0 个删除...va/org/apache/beam/sdk/io/snowflake/SnowflakeService.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeServiceImpl.java 90 个添加, 0 个删除...rg/apache/beam/sdk/io/snowflake/SnowflakeServiceImpl.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/credentials/KeyPairSnowflakeCredentials.java 81 个添加, 0 个删除...io/snowflake/credentials/KeyPairSnowflakeCredentials.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/credentials/OAuthTokenSnowflakeCredentials.java 31 个添加, 0 个删除...snowflake/credentials/OAuthTokenSnowflakeCredentials.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/credentials/SnowflakeCredentials.java 24 个添加, 0 个删除...am/sdk/io/snowflake/credentials/SnowflakeCredentials.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/credentials/SnowflakeCredentialsFactory.java 55 个添加, 0 个删除...io/snowflake/credentials/SnowflakeCredentialsFactory.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/credentials/UsernamePasswordSnowflakeCredentials.java 37 个添加, 0 个删除...ake/credentials/UsernamePasswordSnowflakeCredentials.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/credentials/package-info.java 20 个添加, 0 个删除...pache/beam/sdk/io/snowflake/credentials/package-info.java
- sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/package-info.java 20 个添加, 0 个删除...n/java/org/apache/beam/sdk/io/snowflake/package-info.java
- sdks/java/io/snowflake/src/test/java/org/apache/beam/sdk/io/snowflake/test/FakeSnowflakeBasicDataSource.java 298 个添加, 0 个删除...m/sdk/io/snowflake/test/FakeSnowflakeBasicDataSource.java
- sdks/java/io/snowflake/src/test/java/org/apache/beam/sdk/io/snowflake/test/FakeSnowflakeDatabase.java 81 个添加, 0 个删除...che/beam/sdk/io/snowflake/test/FakeSnowflakeDatabase.java
- sdks/java/io/snowflake/src/test/java/org/apache/beam/sdk/io/snowflake/test/FakeSnowflakeServiceImpl.java 64 个添加, 0 个删除.../beam/sdk/io/snowflake/test/FakeSnowflakeServiceImpl.java
- sdks/java/io/snowflake/src/test/java/org/apache/beam/sdk/io/snowflake/test/TestUtils.java 40 个添加, 0 个删除...java/org/apache/beam/sdk/io/snowflake/test/TestUtils.java
- sdks/java/io/snowflake/src/test/java/org/apache/beam/sdk/io/snowflake/test/package-info.java 20 个添加, 0 个删除...a/org/apache/beam/sdk/io/snowflake/test/package-info.java
加载中
想要评论请 注册 或 登录