* Enable splitting up docker publish workflow

* Just make build.gradle.kts changes for now

* Update slf4j version

* Remove slf4j from arrow dependency exclusion.

* Fix two test failures due to upgrade slf4j-jdk14 to 2.x

The class path of slf4j has been changed from org/slf4j/impl
to org/slf4j/jul.

Two failed tests:
- :runners:google-cloud-dataflow-java:worker:validateShadedJarContainsSlf4jJdk14
- :runners:google-cloud-dataflow-java:worker:validateShadedJarDoesntLeakNonProjectClasses

* Fixed another four failed tests.

The failed tests  are under
org.apache.beam.runners.dataflow.worker.HotKeyLoggerTest

* Bump the default spark version from 3.2.2 to 3.5.0.

The previous version has a compile dependency on slf4j 1.x binding, which would no longer
work with slf4j 2.x.

* Add used but not declared deps for spark 3.5.0

* Temporary modify spark version to 3.x in sparkreceiver.

* Fix failed spark tests.

* A better workaround for Spark 3.2.x

* Take out the add-opens for tests as they were only run in java 8 and 11.

* Mention changes in CHANGES.md

* Update comments

* Move sparkReceiver/2 to sparkreceiver/3 that supports Spark 3.x.

* Minor fix on cdap spark dependency

* Fix Spotless PreCommit

* Fix existing spotless

This reverts commit 4073aba0.

Bumps org.sonarqube from 3.0 to 6.0.0.5145.

---
updated-dependencies:
- dependency-name: org.sonarqube
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove remaining Python 3.8 Artifacts

* Clear container files

* Move TensorRT suite to 3.10

* Vllm first pass [wip]

* Example for integration tests wip

* Still wip

* Test changes

* Dockerfile improvements

* Remove bad change

* Clean up test args

* clean up invocation

* string fix

* string fix

* clean up

* lint

* Get tests working with 5xx driver

* cleanup

* Fixes, everything is now working

* Batching

* lint

* Feedback + CHANGES.md

* Add runners/prism/java to java PreCommit

* revert build.gradle change

* [WIP] Remove Python 3.8 targets from Actions

* revert changes to 3.9 build.gradle

* More gradle cleanup

* Add posargs

* Update local-env-setup.sh

* Fix default env python version

* More places to update python version refs

* mypy/linting fixes

* yapf

* Update other locations

* update more files

* update standard_external_transforms.yaml

* update test case that fails on 3.8 -> 3.9

Co-authored-by: Dilnaz Amanzholova <adilnaz@google.com>

* read integration test

* write integration test

* test with Managed interface

* Don't hardcode python version in tox.ini for lint, mypy and docs.

* fixup

* fixup

Limit elasticsearch testing from core SDK change to just one suite (not full elasticsearch compat suite which is very flaky)

* add terraform for utility cluster. Add name override to gke

* remove whitespace

* add strimzi version and fix prefix

* add more explanation on the purpose

* add node_config block for oauth scopes

* move oauth to auto_provisioning_defaults

* move stimzi operator to separate module. Remove utility cluster module and implement as var file

* add beam-utility tfvars file

* Refactor https://github.com/apache/beam/pull/30847

* add name override. add utility vars and backend

* tfbackend file asf licence

* tfbackend file asf licence

* add readme usage

* remove whitespace

* move kafka to 01-strimzi-operator, rename cluster

* remove beam-utility files

* managed api for java

* yaml utils

* Fix hdfs integration test

debug only

skip yaml test

* Add back azureIntegrationTest get removed when teardown Py37 support

* Clean up java PreCommit

* Move SQL tasks to SQL precommit
  for Java11/17 SQL precommit, still run a small group of tasks as before

* Remove kotlin precommit from Java PreCommit
  Duplicate tests already run in Kotlin Example PreCommit

* Arrange and remove duplicate tasks in javaPreCommit

* Add timeout to java-fn-execution

* Add JvmVerification to sql precommit

* Remove jvmverification for :sqlPreCommit

it is done by ":sdks:java:extensions:sql:preCommit" now

* basic knowledge lookup prompts

* advanced knowledge lookup prompts

* finalize advanced knowledge lookup prompts

* minor content fixes

* pubsub lookup added

* pubsub code generation and explaination added

* initial prompts added

* minor syntax

* fix whitespace

* fix whitespace warnings

* fix code review comments

* remove unfinished work

* fix trailing whitespace

* fix whitespace

* Apply suggestions from code review

Co-authored-by: Rebecca Szper <98840847+rszper@users.noreply.github.com>

* fix code review comments

* Apply suggestions from code review

Co-authored-by: Rebecca Szper <98840847+rszper@users.noreply.github.com>

* added intro text before example

---------

Co-authored-by: Rebecca Szper <98840847+rszper@users.noreply.github.com>

[RRIO] [testing] Downgrade mock-apis grpcio-tools dependencies to remove Beam SDK version conflict  (#29320)

* Constrain grpc* versons to >=1.53.0

* Downgrade grpcio dependencies

* Bump project version

* Make version constraint >=

* Add poetry.lock to rat

* Implement gRPC service with transcoder

* WIP: defer integration tests for later

* Make service executables

* remove grpc gateway

* Clean up and add messy integration test

* Delete flakey integration tests that depend on Docker

* Generate java proto

* Generate python proto

* Python is too much trouble for now

* Finalize README

* Fix rat and whitespace

* Fix whitespace

* Clean up per PR comments

* Fix gcp logging not capturing severity

* Setup Java 21 container (#28833)

* Add Java21 container

* fix spotless

* Update website

* Fix jamm

* align configs in recent change

* incremental fixes

* add more comments for when jpms test enabled

This reverts commit e8e38145.

* Add Java21 container

* fix spotless

* Update website

* Fix jamm

* align configs in recent change

* Updates the Transform Service to accept Python extra packages through the Java API

* Addressing reviewer comments

* Addressing reviewer comments

Bumps org.nosphere.apache.rat from 0.8.0 to 0.8.1.

---
updated-dependencies:
- dependency-name: org.nosphere.apache.rat
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Rearrange Java Example tests on Dataflow

* Fix readme

* Add container using standalone prism go precommit.

* Add GoPrism precommit action.

* Don't serve prism UI by default.

---------

Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com>

* Remove legacy path for Go Dataflow post commits.

* Fix typo.

* Observation: xlang tests aren't run in the Go Dataflow suite.

---------

Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com>

* Add flags to make all containers multi-arch.

* change dockerPush to docker task for other containers.

* update CHANGES.md

* Add GoogleAdsIO for reading from Google Ads

* Remove trailing whitespace

* Add link to OAuth2 documentation in the Google Ads API

* Document retry configuration

* Capitalize long literal suffix in documentation

* Make GoogleAdsV14.Read consume customer IDs from a PCollection

* Change non-empty to non-null in test assertions

* Use Long.toString in transform construction example

* Use EnsuresNonNull, RequiresNonNull instead of checkStateNotNull

* Remove default rate limit policy

* Refactor DefaultRateLimitPolicy to SimpleRateLimitPolicy

* Synchronize on ReadAllFn to safely modify static variable sleeper

* Add additional links to clarify API limits, quotas, and policies

* Fix Guava imports

* Update CHANGES.md

* Move change to upcoming release

* Setup Java Flink runner PreCommit

* Split Flink 1.13-1.15 tests from main Java PreCommit

* Only run one flink version on main java precommit

* address comments

* set timeout

* address comment

* address comments

* Fix copy-paste

* Disable spotless and checkstyle