* Fix pipeline translate for Spark3

* Refactoring

* RecordWriter change

* added metricsConfig to RecordWriter

* formatting

* removed comments

* fixed tests

* changed tests

* removed a comment

* 2 to 3

* Add yaml examples generation to the release scripts.

* Update .github/workflows/build_release_candidate.yml

Co-authored-by: Danny McCormick <dannymccormick@google.com>

---------

Co-authored-by: Danny McCormick <dannymccormick@google.com>

* Putting the default retry settings for Spanner which was overridden in Dataflow templates

* addressing comments

* Spotless apply

* Correcting checkstyle violation

* Bump dataflow java container version

* make them the same because of #30634

* Add main and auxiliary transforms.

* Minor fix per reviewer's feedback and fix lints.

This fixes a cache pollution bug.

This will make it easier to accommodate packaging schemes that
do not place the data files next to the python files.

* Fix docker dev environment set up

* Remove python 3.6 alias in Docker dev

* call out the snappy version in the warning message

* Update sdks/python/apache_beam/io/tfrecordio.py

Co-authored-by: tvalentyn <tvalentyn@users.noreply.github.com>

---------

Co-authored-by: tvalentyn <tvalentyn@users.noreply.github.com>

use 2025.1.1

Co-authored-by: Naireen <naireenhussain@google.com>

* Pin python-snappy==0.6.1

* fixed the workflow to install libsnappy-dev

* Add detectors and some necessary univariate trackers.

* Make score_one return float or None.

* Raise exception when more than one variable used in univariate detectors.

* Fix lints.

* implement a clickhouse resource manager

* create integration test for ClickHouseResourceManager

* create unit tests for ClickHouseResourceManager

* fix warnings

* address cr comments

* fix mixed junit4 and 5 imports

* arrange import

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.4...3.1.6

)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Skip some tests on Dataflow runners

* run spotless

* changed to assumeFalse

* format comments

* add message to assumeFalse

Using windows sets does not allow us to correctly compute timestamps
for the resulting element when there is more than one window.
This is particularly problematic for runners (such as Dataflow)
that base watermarks (and window firings) on these timestamps
by arranging that elements are seen in timstamp sorted order (e.g.
by using the timestamp as a secondary key).

* Fix test.

* Trigger post commit.

* Use highmem22 for postcommit python.

* Run postcommit 311.

* Revert "Run postcommit 311."

This reverts commit a3de98f723b14865086aead0306f5c324514252e.

* Use flink config in portable tests.

* Temporarily disable some tests.

* Revert "Temporarily disable some tests."

This reverts commit e3ae832926928ea72d8758b9323cc3bb9420c671.

* Move xlang_jdbc to xlang direct.

* Trigger affected workflows.

---------

Co-authored-by: Claude <cvandermerwe@google.com>

* Increase timeout for test

* Decrease num futures in test

* Catch union-of-iterables case in get_yielded_type()

* add test case for mixed union

* Add AlloyDB language connector support.

* Add test.

* Trigger test.

* Add link to WriteToJdbc.

---------

Co-authored-by: Claude <cvandermerwe@google.com>

* Correctly parse labels if they are passed as a single string instead of a list

* Fix dict case adnd add tests

* format

* Created using Colab

* Created using Colab

* Remove extra link.