标签

0.9.0.1 release

Confluent Platform 2.0.0 Release

0.9.0.0 release

0.8.2.2 release

0.8.2.0-cp release

0.8.2.1 release

For 0.8.2.0 release.

0.8.2.0 release

New Feature
[KAFKA-1227] - Code dump of new producer
[KAFKA-1384] - Log Broker state
[KAFKA-1443] - Add delete topic to topic commands and update DeleteTopicCommand
[KAFKA-1512] - Limit the maximum number of connections per ip address

Task
[KAFKA-1399] - Drop Scala 2.8.x support
[KAFKA-1559] - Upgrade Gradle wrapper to Gradle 2.0

Test
[KAFKA-1079] - Liars in PrimitiveApiTest that promise to test api in compression mode, but don't do this actually
[KAFKA-1285] - enable log4j in unit test
[KAFKA-1411] - remove unused test cases

Wish
[KAFKA-1308] - Publish jar of test utilities to Maven
[KAFKA-1370] - Gradle startup script for Windows
[KAFKA-1371] - Ignore build output dirs
[KAFKA-1485] - Upgrade to Zookeeper 3.4.6 and create "shim" for ZKCLI so system tests can run

Sub-task
[KAFKA-784] - creating topic without partitions, deleting then creating with partition causes errors in 'kafka-list-topic'
[KAFKA-1240] - Add ability to existing system tests to use the new producer client
[KAFKA-1304] - unregistered mbean exception in new producer
[KAFKA-1316] - Refactor Sender
[KAFKA-1328] - Add new consumer APIs
[KAFKA-1359] - Add topic/broker metrics once new topic/broker is discovered
[KAFKA-1462] - Add new request and response formats for the new consumer and coordinator communication
[KAFKA-1472] - Add the compression ratio metrics in the new producer
[KAFKA-1611] - Improve system test configuration

Improvement
[KAFKA-404] - When using chroot path, create chroot on startup if it doesn't exist
[KAFKA-997] - Provide a strict verification mode when reading configuration properties
[KAFKA-1028] - per topic configuration of preference for consistency over availability
[KAFKA-1204] - Add KAFKA_LOG_DIR to allow LOG_DIR to be outside of code dir
[KAFKA-1264] - Make ConsoleProducer compression codec configurable
[KAFKA-1278] - More flexible helper scripts
[KAFKA-1287] - enable log4j in command line tools using the new producer
[KAFKA-1291] - Make wrapper shell scripts for important tools
[KAFKA-1298] - Controlled shutdown tool doesn't seem to work out of the box
[KAFKA-1312] - Update gitignore to ignore IDE generated files
[KAFKA-1325] - Fix inconsistent per topic log configs
[KAFKA-1344] - Kafka-console-producer.sh support snappy compression
[KAFKA-1348] - Producer's Broker Discovery Interface
[KAFKA-1353] - report capacity used by request thread pool and network thread pool
[KAFKA-1361] - enable controlled shutdown by default
[KAFKA-1390] - TestUtils.waitUntilLeaderIsElectedOrChanged may wait longer than it needs
[KAFKA-1392] - all TestUtiles.waitUntilTrue() should throw an exception if the return value is false
[KAFKA-1414] - Speedup broker startup after hard reset
[KAFKA-1419] - cross build for scala 2.11
[KAFKA-1425] - provide logStartOffset in addition to LogEndOffset metric
[KAFKA-1430] - Purgatory redesign
[KAFKA-1446] - Consumer metrics for rebalance
[KAFKA-1454] - Update for a Scala 2.11 build
[KAFKA-1456] - Add LZ4 and LZ4C as a compression codec
[KAFKA-1480] - support log.roll.minutes
[KAFKA-1488] - new metrics for measuring the ratio when the new producer is block for space allocation
[KAFKA-1493] - Use a well-documented LZ4 compression format and remove redundant LZ4HC option
[KAFKA-1513] - Cleanup KafkaServerStartable code
[KAFKA-1519] - Console consumer: expose configuration option to enable/disable writing the line separator
[KAFKA-1535] - return all live brokers in TopicMetadataResponse
[KAFKA-1555] - provide strong consistency with reasonable availability
[KAFKA-1576] - Make "deleteTopic" a bit more user friendly
[KAFKA-1618] - Exception thrown when running console producer with no port number for the broker
[KAFKA-1620] - Make kafka api protocol implementation public
[KAFKA-1644] - Inherit FetchResponse from RequestOrResponse

Bug
[KAFKA-179] - Log files always touched when broker is bounced
[KAFKA-294] - "Path length must be > 0" error during startup
[KAFKA-472] - update metadata in batches in Producer
[KAFKA-703] - A fetch request in Fetch Purgatory can double count the bytes from the same delayed produce request
[KAFKA-847] - kafka appender layout does not work for kafka 0.7.1
[KAFKA-899] - LeaderNotAvailableException the first time a new message for a partition is processed.
[KAFKA-924] - Specify console consumer properties via a single --property command line parameter
[KAFKA-1002] - Delete aliveLeaders field from LeaderAndIsrRequest
[KAFKA-1025] - Producer.send should provide recoverability info on failiure
[KAFKA-1026] - Dynamically Adjust Batch Size Upon Receiving MessageSizeTooLargeException
[KAFKA-1034] - Improve partition reassignment to optimize writes to zookeeper
[KAFKA-1041] - Number of file handles increases indefinitely in producer if broker host is unresolvable
[KAFKA-1061] - Break-down sendTime to multipleSendTime
[KAFKA-1096] - An old controller coming out of long GC could update its epoch to the latest controller's epoch
[KAFKA-1108] - when controlled shutdown attempt fails, the reason is not always logged
[KAFKA-1123] - Broker IPv6 addresses parsed incorrectly
[KAFKA-1146] - toString() on KafkaStream gets stuck indefinitely
[KAFKA-1147] - Consumer socket timeout should be greater than fetch max wait
[KAFKA-1179] - createMessageStreams() in javaapi.ZookeeperConsumerConnector does not throw
[KAFKA-1180] - WhiteList topic filter gets a NullPointerException on complex Regex
[KAFKA-1189] - kafka-server-stop.sh doesn't stop broker
[KAFKA-1192] - Enable DumpLogSegments tool to deserialize messages
[KAFKA-1193] - Data loss if broker is killed using kill -9
[KAFKA-1209] - System test transient failure due to ProducerPerformance's connection to servers kept broken
[KAFKA-1210] - Windows Bat files are not working properly
[KAFKA-1212] - System test exception handling does not stop background producer threads
[KAFKA-1213] - Adding fetcher needs to be avoided upon make-follower when replica manager is shutting down
[KAFKA-1238] - New producer hangs in a loop detecting metadata for auto created topics
[KAFKA-1276] - Provide a list of config overrides available when running kafka.topics
[KAFKA-1280] - exclude kafka-clients jar from dependant-libs dir
[KAFKA-1297] - releaseTarGz target needs signing task
[KAFKA-1301] - system testcase_0206 fails using the new producer
[KAFKA-1302] - cleanup logging in new producer
[KAFKA-1305] - Controller can hang on controlled shutdown with auto leader balance enabled
[KAFKA-1307] - potential socket leak in new producer and clean up
[KAFKA-1318] - waiting for producer to stop is not reliable in system tests
[KAFKA-1319] - kafka jar doesn't depend on metrics-annotation any more
[KAFKA-1364] - ReplicaManagerTest hard-codes log dir
[KAFKA-1369] - snappy version update 1.1.x
[KAFKA-1375] - Formatting for "Running a task on a particular version of Scala" paragraph in README.md is broken
[KAFKA-1376] - transient test failure in UncleanLeaderElectionTest
[KAFKA-1378] - transient unit test failure in LogRecoveryTest
[KAFKA-1381] - transient unit test failure in AddPartitionsTest
[KAFKA-1382] - Update zkVersion on partition state update failures
[KAFKA-1383] - transient unit test failure in SocketServerTest
[KAFKA-1389] - transient unit test failure in ProducerFailureHandlingTest
[KAFKA-1391] - transient unit test failure in DeleteTopicTest.testPreferredReplicaElectionDuringDeleteTopic
[KAFKA-1395] - fix unit tests in AutoOffsetResetTest
[KAFKA-1396] - fix transient unit test ProducerFailureHandlingTest.testBrokerFailure
[KAFKA-1397] - delete topic is not working
[KAFKA-1400] - transient unit test failure in SocketServerTest
[KAFKA-1406] - Fix scaladoc/javadoc warnings
[KAFKA-1409] - oversized messages can slow down the brokers
[KAFKA-1410] - MetadataCache cleanup
[KAFKA-1412] - transient unit test failure in ProducerSendTest.testAutoCreateTopic
[KAFKA-1418] - transient unit test failure in ProducerFailureHandlingTest
[KAFKA-1422] - Fix transient unit test SocketServerTest
[KAFKA-1424] - transient unit test failure in testSendWithDeadBroker
[KAFKA-1431] - ConsoleConsumer - Option to clean zk consumer path
[KAFKA-1432] - Make num.producerThreads configurable on new MirrrorMaker
[KAFKA-1433] - transient unit test failure in ZookeeperConsumerConnectorTest
[KAFKA-1437] - ConsumerMetadataResponse should always include coordinator information
[KAFKA-1438] - Migrate kafka client tools
[KAFKA-1439] - transient unit test failure in testDeleteTopicDuringAddPartition
[KAFKA-1442] - RBTools post-review is deprecated
[KAFKA-1444] - kafka.javaapi.TopicMetadata and PartitionMetadata doesn't forward the toString method
[KAFKA-1451] - Broker stuck due to leader election race
[KAFKA-1453] - Add a channel queue jmx in Mirror Maker
[KAFKA-1459] - kafka.tools.ConsumerOffsetChecker throws NoNodeException
[KAFKA-1468] - Improve perf tests
[KAFKA-1469] - Util.abs function does not return correct absolute values for negative values
[KAFKA-1473] - transient unit test failure in testRequestHandlingDuringDeleteTopic
[KAFKA-1482] - Transient test failures for kafka.admin.DeleteTopicTest
[KAFKA-1487] - add test jars to gradle build for packaging and release
[KAFKA-1490] - remove gradlew initial setup output from source distribution
[KAFKA-1491] - ConsumerMetadataResponse is not read completely
[KAFKA-1502] - source jar is empty
[KAFKA-1503] - all partitions are using same broker as their leader after broker is down
[KAFKA-1510] - Force offset commits when migrating consumer offsets from zookeeper to kafka
[KAFKA-1511] - Malformated link and consumer/producer mixup in documentation
[KAFKA-1529] - transient unit test failure in testAutoCreateAfterDeleteTopic
[KAFKA-1531] - zookeeper.connection.timeout.ms is set to 10000000 in configuration file in Kafka tarball
[KAFKA-1533] - transient unit test failure in ProducerFailureHandlingTest
[KAFKA-1539] - Due to OS caching Kafka might loose offset files which causes full reset of data
[KAFKA-1542] - normal IOException in the new producer is logged as ERROR
[KAFKA-1544] - LogCleaner may take a long time to shutdown
[KAFKA-1549] - dead brokers coming in the TopicMetadataResponse
[KAFKA-1557] - ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth)
[KAFKA-1558] - AdminUtils.deleteTopic does not work
[KAFKA-1562] - kafka-topics.sh alter add partitions resets cleanup.policy
[KAFKA-1567] - Metric memory leaking after closing the clients
[KAFKA-1571] - MetadataeTest hangs
[KAFKA-1574] - unit tests can hang on socketserver shutdown
[KAFKA-1577] - Exception in ConnectionQuotas while shutting down
[KAFKA-1578] - Controller should de-register all listeners upon designation
[KAFKA-1585] - Client: Infinite "conflict in /consumers/"
[KAFKA-1598] - Fix variable typo in Kafka main class
[KAFKA-1600] - Controller failover not working correctly.
[KAFKA-1603] - MirrorMaker consumer does not put the message key into ProducerRecord
[KAFKA-1609] - New producer metadata response handling should only exclude a PartitionInfo when its error is LEADER_NOT_AVAILABLE
[KAFKA-1616] - Purgatory Size and Num.Delayed.Request metrics are incorrect
[KAFKA-1619] - perf dir can be removed
[KAFKA-1622] - project shouldn't require signing to build
[KAFKA-1635] - Java doc of makeLeaders in ReplicaManager is wrong
[KAFKA-1645] - some more jars in our src release
[KAFKA-1662] - gradle release issue permgen space
[KAFKA-1663] - Controller unable to shutdown after a soft failure
[KAFKA-1670] - Corrupt log files for segment.bytes values close to Int.MaxInt
[KAFKA-1671] - uploaded archives are missing for Scala version 2.11
[KAFKA-1673] - potential java.lang.IllegalStateException in BufferPool.allocate()
[KAFKA-1681] - Newly elected KafkaController might not start deletion of pending topics
[KAFKA-1699] - autoRebalanceScheduler.shutdown() causes deadlock while controller shutting down
[KAFKA-1717] - remove netty dependency through ZK 3.4.x

Bug
[KAFKA-1310] - Zookeeper timeout causes deadlock in Controller
[KAFKA-1311] - Add a flag to turn off delete topic until it is stable
[KAFKA-1315] - log.dirs property in KafkaServer intolerant of trailing slash
[KAFKA-1317] - KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[KAFKA-1323] - log.dirs server property no longer supports relative directories
[KAFKA-1350] - Fix excessive state change logging
[KAFKA-1355] - Reduce/optimize update metadata requests sent during leader election
[KAFKA-1358] - Broker throws exception when reconnecting to zookeeper
[KAFKA-1362] - Publish sources and javadoc jars
[KAFKA-1363] - testTopicConfigChangesDuringDeleteTopic hangs
[KAFKA-1365] - Second Manual preferred replica leader election command always fails
[KAFKA-1373] - LogCleaner assumes first dirty offset zero if there is no cleaner checkpoint
[KAFKA-1398] - Topic config changes can be lost and cause fatal exceptions on broker restarts

Task
[KAFKA-1413] - bump kafka version to 0.8.1.1 in gradle.properties

0.8.1 release

Release Notes - Kafka - Version 0.8.1

** New Feature
    * [KAFKA-554] - Move all per-topic configuration into ZK and add to the CreateTopicCommand
    * [KAFKA-561] - Rebuild index file for a log segment if there is none
    * [KAFKA-657] - Add an API to commit offsets
    * [KAFKA-918] - Change log.retention.hours to be log.retention.mins
    * [KAFKA-925] - Add optional partition key override in producer
    * [KAFKA-1092] - Add server config parameter to separate bind address and ZK hostname
    * [KAFKA-1117] - tool for checking the consistency among replicas

** Improvement
    * [KAFKA-347] - change number of partitions of a topic online
    * [KAFKA-741] - Improve log cleaning dedupe buffer efficiency
    * [KAFKA-1001] - Handle follower transition in batch
    * [KAFKA-1084] - Validate properties for custom serializers
    * [KAFKA-1127] - kafka and zookeeper server should start in daemon mode and log to correct position
    * [KAFKA-1131] - copy some more files into the release tar and zip that are needed/desired
    * [KAFKA-1136] - Add subAppend in Log4jAppender for generic usage
    * [KAFKA-1158] - remove bin/run-rat.sh
    * [KAFKA-1159] - try to get the bin tar smaller
    * [KAFKA-1160] - have the pom reference the exclusions necessary so folks don't have to
    * [KAFKA-1161] - review report of the dependencies, conflicts, and licenses into ivy-report
    * [KAFKA-1162] - handle duplicate entry for ZK in the pom file
    * [KAFKA-1163] - see whats going on with the 2.8.0 pom
    * [KAFKA-1186] - Add topic regex to the kafka-topics tool
    * [KAFKA-1232] - make TopicCommand more consistent

** Bug
    * [KAFKA-172] - The existing perf tools are buggy
    * [KAFKA-184] - Log retention size and file size should be a long
    * [KAFKA-330] - Add delete topic support
    * [KAFKA-515] - Log cleanup can close a file channel opnened by Log.read before the transfer completes
    * [KAFKA-615] - Avoid fsync on log segment roll
    * [KAFKA-648] - Use uniform convention for naming properties keys
    * [KAFKA-671] - DelayedProduce requests should not hold full producer request data
    * [KAFKA-677] - Retention process gives exception if an empty segment is chosen for collection
    * [KAFKA-712] - Controlled shutdown tool should provide a meaningful message if a controller failover occurs during the operation
    * [KAFKA-739] - Handle null values in Message payload
    * [KAFKA-759] - Commit/FetchOffset APIs should not return versionId
    * [KAFKA-852] - Remove clientId from OffsetFetchResponse and OffsetCommitResponse
    * [KAFKA-897] - NullPointerException in ConsoleConsumer
    * [KAFKA-930] - Integrate preferred replica election logic into kafka
    * [KAFKA-933] - Hadoop example running DataGenerator causes kafka.message.Message cannot be cast to [B exception
    * [KAFKA-985] - Increasing log retention quickly overflows scala Int
    * [KAFKA-1004] - Handle topic event for trivial whitelist topic filters
    * [KAFKA-1009] - DumpLogSegments tool should return error on non-existing files
    * [KAFKA-1020] - Remove getAllReplicasOnBroker from KafkaController
    * [KAFKA-1036] - Unable to rename replication offset checkpoint in windows
    * [KAFKA-1052] - integrate add-partitions command into topicCommand
    * [KAFKA-1055] - BrokerTopicStats is updated before checking for MessageSizeTooLarge
    * [KAFKA-1060] - Break-down sendTime into responseQueueTime and the real sendTime
    * [KAFKA-1074] - Reassign partitions should delete the old replicas from disk
    * [KAFKA-1090] - testPipelinedRequestOrdering has transient failures
    * [KAFKA-1091] - full topic list can be read from metadata cache in the broker instead of ZK
    * [KAFKA-1097] - Race condition while reassigning low throughput partition leads to incorrect ISR information in zookeeper
    * [KAFKA-1103] - Consumer uses two zkclients
    * [KAFKA-1104] - Consumer uses two zkclients
    * [KAFKA-1112] - broker can not start itself after kafka is killed with -9
    * [KAFKA-1116] - Need to upgrade sbt-assembly to compile on scala 2.10.2
    * [KAFKA-1121] - DumpLogSegments tool should print absolute file name to report inconsistencies
    * [KAFKA-1128] - Github is still showing 0.7 as the default branch
    * [KAFKA-1129] - if we have a script to run the jar then we should include it in the build or remove it during release
    * [KAFKA-1133] - LICENSE and NOTICE files need to get into  META-INF when jars are built before they're signed for publishing to maven
    * [KAFKA-1135] - Code cleanup - use Json.encode() to write json data to zk
    * [KAFKA-1140] - Move the decoding logic from ConsumerIterator.makeNext to next
    * [KAFKA-1141] - make changes to downloads for the archive old releases to new old_releases folder
    * [KAFKA-1151] - The Hadoop consumer API doc is not referencing the contrib consumer
    * [KAFKA-1152] - ReplicaManager's handling of the leaderAndIsrRequest should gracefully handle leader == -1
    * [KAFKA-1154] - replicas may not have consistent data after becoming follower
    * [KAFKA-1156] - Improve reassignment tool to output the existing assignment to facilitate rollbacks
    * [KAFKA-1157] - Clean up Per-topic Configuration from Kafka properties
    * [KAFKA-1164] - kafka should depend on snappy 1.0.5 (instead of 1.0.4.1)
    * [KAFKA-1166] - typo in kafka-server-stop.sh
    * [KAFKA-1168] - OfflinePartitionCount in JMX can be incorrect during controlled shutdown
    * [KAFKA-1169] - missing synchronization in access to leaderCache in KafkaApis
    * [KAFKA-1170] - ISR can be inconsistent during partition reassignment for low throughput partitions
    * [KAFKA-1172] - error when creating topic with kafka-topics.sh
    * [KAFKA-1183] - DefaultEventHandler causes unbalanced distribution of messages across partitions
    * [KAFKA-1198] - NullPointerException in describe topic
    * [KAFKA-1200] - inconsistent log levels when consumed offset is reset
    * [KAFKA-1202] - optimize ZK access in KafkaController
    * [KAFKA-1205] - README in examples not update
    * [KAFKA-1208] - Update system test still to use kafka-topics instead of kafka-create-topics shell
    * [KAFKA-1214] - Support arguments to zookeeper-shell.sh script
    * [KAFKA-1228] - Socket Leak on ReplicaFetcherThread
    * [KAFKA-1243] - Gradle issues for release
    * [KAFKA-1263] - Snazzy up the README markdown for better visibility on github
    * [KAFKA-1271] - controller logs exceptions during ZK session expiration
    * [KAFKA-1275] - fixes for quickstart documentation

** Task
    * [KAFKA-823] - merge 0.8 (51421fcc0111031bb77f779a6f6c00520d526a34) to trunk
    * [KAFKA-896] - merge 0.8 (988d4d8e65a14390abd748318a64e281e4a37c19) to trunk
    * [KAFKA-965] - merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk
    * [KAFKA-1051] - merge from 0.8 da4512174b6f7c395ffe053a86d2c6bb19d2538a to trunk
    * [KAFKA-1080] - why are builds for 2.10 not coming out with the trailing minor version number

** Sub-task
    * [KAFKA-121] - pom should include standard maven niceties
    * [KAFKA-1244] - The LICENSE and NOTICE are missing from the jar files
    * [KAFKA-1245] - the jar files and pom are not being signed so nexus is failing to publish them
    * [KAFKA-1246] - The 2.10 version is showing up as 2.10.1
    * [KAFKA-1248] - jars are missing from maven upload that were previously there
    * [KAFKA-1249] - release tar name is different than 0.8.0
    * [KAFKA-1254] - remove vestigial sbt
    * [KAFKA-1274] - gradle.properties needs the variables used in the build.gradle

New Feature
[KAFKA-50] - kafka intra-cluster replication support
[KAFKA-188] - Support multiple data directories
[KAFKA-202] - Make the request processing in kafka asynchonous
[KAFKA-203] - Improve Kafka internal metrics
[KAFKA-235] - Add a 'log.file.age' configuration parameter to force rotation of log files after they've reached a certain age
[KAFKA-429] - Expose JMX operation to set logger level dynamically
[KAFKA-475] - Time based log segment rollout
[KAFKA-545] - Add a Performance Suite for the Log subsystem
[KAFKA-546] - Fix commit() in zk consumer for compressed messages
Improvement
[KAFKA-100] - ProducerShell should use high-level producer instead of SyncProducer
[KAFKA-133] - Publish kafka jar to a public maven repository
[KAFKA-139] - cross-compile multiple Scala versions and upgrade to SBT 0.12.1
[KAFKA-155] - Support graceful Decommissioning of Broker
[KAFKA-165] - Add helper script for zkCli.sh
[KAFKA-181] - Log errors for unrecognized config options
[KAFKA-187] - Add Snappy Compression as a Codec and refactor CompressionUtil and option on startup to select what the default codec
[KAFKA-193] - use by name parameter helper for logging and trait to include lazy logging and refactor code to use the new LogHelper
[KAFKA-246] - log configuration values used
[KAFKA-253] - Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster
[KAFKA-258] - Remove broker.id from the broker.list config in the Producer
[KAFKA-267] - Enhance ProducerPerformance to generate unique random Long value for payload
[KAFKA-271] - Modify new FetchResponse object to remove the initial offset field
[KAFKA-281] - support multiple root log directories
[KAFKA-285] - Increase maximum value of log.retention.size
[KAFKA-314] - Go Client Multi-produce
[KAFKA-323] - Add the ability to use the async producer in the Log4j appender
[KAFKA-324] - enforce broker.id to be a non-negative integer
[KAFKA-325] - revisit broker config in 0.8
[KAFKA-349] - Create individual "Response" types for each kind of request and wrap them with "BoundedByteBufferSend", remove "xxResponseSend" types for all requests except "FetchRequest"
[KAFKA-365] - change copyright in NOTICE to 2012
[KAFKA-366] - add jmx beans in broker to track # bytes in consumer
[KAFKA-368] - use the pig core jar from maven instead of distributing it
[KAFKA-393] - Add constructor for message which takes both byte array offset and length
[KAFKA-408] - ProducerPerformance does not work with all producer config options
[KAFKA-437] - Unused var statement in ZookeeperConsumerConnectorTest
[KAFKA-439] - @returns was used in scala doc when it should have been @return
[KAFKA-505] - Remove errorcode from TopicMetaDataResponse
[KAFKA-548] - remove partition from ProducerRequestPartitionData and FetchResponsePartitionData
[KAFKA-581] - provides windows batch script for starting Kafka/Zookeeper
[KAFKA-632] - ProducerRequest should take ByteBufferMessageSet instead of MessageSet
[KAFKA-638] - remove ProducerShell
[KAFKA-667] - Rename .highwatermark file
[KAFKA-675] - Only bind to the interface declared in the 'hostname' config property
[KAFKA-699] - Disallow clients to set replicaId in FetchRequest
[KAFKA-762] - Improve second replica assignment
[KAFKA-763] - Add an option to replica from the largest offset during unclean leader election
[KAFKA-812] - Support deep iteration in DumpLogSegments tool
[KAFKA-850] - add an option to show under replicated partitions in list topic command
[KAFKA-923] - Improve controller failover latency
[KAFKA-931] - make zookeeper.connect a required property
[KAFKA-1046] - Added support for Scala 2.10 builds while maintaining compatibility with 2.8.x
Task
[KAFKA-93] - Change code header to follow standard ASF source header
[KAFKA-317] - Add support for new wire protocol to Go client
[KAFKA-341] - Create a new single host system test to validate all replicas on 0.8 branch
[KAFKA-348] - rebase 0.8 branch from trunk
[KAFKA-380] - Enhance single_host_multi_brokers test with failure to trigger leader re-election in replication
[KAFKA-440] - Create a regression test framework for distributed environment testing
[KAFKA-526] - System Test should remove the top level data log directory
[KAFKA-594] - Update System Test due to new argument "--sync" in ProducerPerformance
[KAFKA-605] - System Test - Log Retention Cases should wait longer before getting the common starting offset in replica log segments
[KAFKA-688] - System Test - Update all testcase_xxxx_properties.json for properties keys uniform naming convention
[KAFKA-737] - System Test : Disable shallow.iterator in Mirror Maker test cases to make compression work correctly
[KAFKA-791] - Fix validation bugs in System Test
[KAFKA-792] - Update multiple attributes in testcase_xxxx_properties.json
[KAFKA-819] - System Test : Add validation of log segment index to offset
[KAFKA-883] - System Test - update migration tool testsuite after 0.7 ProducerPerformance sends seq MessageID
[KAFKA-922] - System Test - set retry.backoff.ms=300 to testcase_0119
[KAFKA-932] - System Test - set retry.backoff.ms=300 to all test cases
Sub-task
[KAFKA-176] - Fix existing perf tools
[KAFKA-237] - create/delete ZK path for a topic in an admin tool
[KAFKA-239] - Wire existing producer and consumer to use the new ZK data structure
[KAFKA-240] - implement new producer and consumer request format
[KAFKA-329] - Remove the watches/broker for new topics and partitions and change create topic admin API to send start replica state change to all brokers
[KAFKA-335] - Implement an embedded controller
[KAFKA-336] - add an admin RPC to communicate state changes between the controller and the broker
[KAFKA-337] - upgrade ZKClient to allow conditional updates in ZK
[KAFKA-338] - controller failover
[KAFKA-339] - using MultiFetch in the follower
[KAFKA-342] - revisit the broker startup procedure according to V3 design
[KAFKA-343] - revisit the become leader and become follower state change operations using V3 design
[KAFKA-344] - migration tool from 0.7 to 0.8
[KAFKA-356] - Create a generic Kafka thread class that includes basic boiler plate code of instantiating and shutting down threads cleanly
[KAFKA-362] - ZookeeperConsumerConnector needs to connect to new leader after leadership change
[KAFKA-369] - remove ZK dependency on producer
[KAFKA-458] - remove errorcode from ByteBufferMessageSet
[KAFKA-482] - Make producer to run for the entire duration of the System Test
[KAFKA-488] - Port Mirroring System Test to this python system test framework
[KAFKA-492] - Sometimes the python system test framework doesn't terminate all running processes
[KAFKA-494] - Relative paths should be used for svg URLs in dashboards html
[KAFKA-502] - Simplify setup / initialization in replication_basic_test.py
[KAFKA-503] - Support "testcase_to_run" or "testcase_to_skip"
[KAFKA-507] - Shut down ZK last to avoid hanging brokers running processes
[KAFKA-513] - Add state change log to Kafka brokers
[KAFKA-571] - Add more test cases to System Test
[KAFKA-731] - ~/ivy2/cache should be a variable in the kafka-run-class bash script
[KAFKA-780] - Reassign partitions tool produces NPE in shutdown handler
[KAFKA-814] - Controller should not throw exception when a preferred replica is already the leader for a partition
[KAFKA-843] - Re-add the release-zip sbt target
Bug
[KAFKA-15] - SBT release-zip target doesn't include bin and config directories anymore
[KAFKA-42] - Support rebalancing the partitions with replication
[KAFKA-43] - Rebalance to preferred broke with intra-cluster replication support
[KAFKA-46] - Commit thread, ReplicaFetcherThread for intra-cluster replication
[KAFKA-49] - Add acknowledgement to the produce request.
[KAFKA-81] - wrong path in bin/kafka-run-class.sh
[KAFKA-97] - SocketServer.scala refers to Handler-specific variables
[KAFKA-171] - Kafka producer should do a single write to send message sets
[KAFKA-192] - CompressionUtilTest does not run and fails when it does
[KAFKA-229] - SimpleConsumer is not logging exceptions correctly so detailed stack trace is not coming in the logs
[KAFKA-233] - The producer's load balancing logic can send requests to dead brokers, when using the async producer option
[KAFKA-259] - Give better error message when trying to run shell scripts without having built/downloaded the jars yet
[KAFKA-295] - Bug in async producer DefaultEventHandler retry logic
[KAFKA-305] - SyncProducer does not correctly timeout
[KAFKA-306] - broker failure system test broken on replication branch
[KAFKA-351] - Refactor some new components introduced for replication
[KAFKA-352] - Throw exception to client if it makes a produce/consume request to a Kafka broker for a topic that hasn't been created
[KAFKA-367] - StringEncoder/StringDecoder use platform default character set
[KAFKA-370] - Exception "java.util.NoSuchElementException: None.get" appears inconsistently in Mirror Maker log.
[KAFKA-371] - Creating topic of empty string puts broker in a bad state
[KAFKA-376] - expose different data to fetch requests from the follower replicas and consumer clients
[KAFKA-379] - TopicCount.constructTopicCount isn't thread-safe
[KAFKA-385] - RequestPurgatory enhancements - expire/checkSatisfy issue; add jmx beans
[KAFKA-386] - Race condition in accessing ISR
[KAFKA-391] - Producer request and response classes should use maps
[KAFKA-396] - Mirroring system test fails on 0.8
[KAFKA-412] - deal with empty TopicData list in producer and fetch request
[KAFKA-413] - single_host_multi_brokers system test fails on laptop
[KAFKA-415] - Controller throws NoSuchElementException while marking a broker failed
[KAFKA-416] - Controller tests throw several zookeeper errors
[KAFKA-417] - NullPointerException in ZookeeperConsumerConnector
[KAFKA-418] - NullPointerException in ConsumerFetcherManager
[KAFKA-420] - maintain HW correctly with only 1 replica in ISR
[KAFKA-422] - LazyInitProducerTest has transient test failure
[KAFKA-424] - Remove invalid mirroring arguments from kafka-server-start.sh
[KAFKA-425] - Wrong class name in performance test scripts
[KAFKA-427] - LogRecoverTest.testHWCheckpointWithFailuresSingleLogSegment has transient failure
[KAFKA-428] - need to update leaderAndISR path in ZK conditionally in ReplicaManager
[KAFKA-431] - LogCorruptionTest.testMessageSizeTooLarge fails occasionally
[KAFKA-433] - ensurePartitionLeaderOnThisBroker should not access ZK
[KAFKA-434] - IllegalMonitorStateException in ReplicaManager.makerFollower
[KAFKA-453] - follower replica may need to backoff the fetching if leader is not ready yet
[KAFKA-456] - ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation
[KAFKA-459] - KafkaController.RequestSendThread can throw exception on broker socket
[KAFKA-460] - ControllerChannelManager needs synchronization btw shutdown and add/remove broker
[KAFKA-461] - remove support for format for magic byte 0 in 0.8
[KAFKA-463] - log.truncateTo needs to handle targetOffset smaller than the lowest offset in the log
[KAFKA-464] - KafkaController NPE in SessionExpireListener
[KAFKA-466] - Controller tests throw IllegalStateException
[KAFKA-467] - Controller based leader election failed ERROR messages in LazyInitProducerTest
[KAFKA-468] - String#getBytes is platform dependent
[KAFKA-470] - transient unit test failure in RequestPurgatoryTest
[KAFKA-471] - Transient failure in ProducerTest
[KAFKA-473] - Use getMetadata Api in ZookeeperConsumerConnector
[KAFKA-474] - support changing host/port of a broker
[KAFKA-481] - Require values in Utils.getTopic* methods to be positive
[KAFKA-490] - Check max message size on server instead of producer
[KAFKA-491] - KafkaRequestHandler needs to handle exceptions
[KAFKA-495] - Handle topic names with "/" on Kafka server
[KAFKA-497] - recover consumer during unclean leadership change
[KAFKA-499] - Refactor controller state machine
[KAFKA-500] - javaapi support for getTopoicMetaData
[KAFKA-501] - getOfffset Api needs to return different latest offset to regular and follower consumers
[KAFKA-506] - Store logical offset in log
[KAFKA-508] - split out partiondata from fetchresponse and producerrequest
[KAFKA-509] - server should shut down on encountering invalid highwatermark file
[KAFKA-510] - broker needs to know the replication factor per partition
[KAFKA-511] - offset returned in Producer response may not be correct
[KAFKA-512] - Remove checksum from ByteBufferMessageSet.iterator
[KAFKA-514] - Replication with Leader Failure Test: Log segment files checksum mismatch
[KAFKA-516] - Consider catching all exceptions in ShutdownableThread
[KAFKA-525] - newly created partitions are not added to ReplicaStateMachine
[KAFKA-528] - IndexOutOfBoundsException thrown by kafka.consumer.ConsumerFetcherThread
[KAFKA-531] - kafka.server.ReplicaManager: java.nio.channels.NonWritableChannelException
[KAFKA-537] - expose clientId and correlationId in ConsumerConfig
[KAFKA-539] - Replica.hw should be initialized to the smaller of checkedpointed HW and log end offset
[KAFKA-540] - log.append() should halt on IOException
[KAFKA-553] - confusing reference to zk.connect in config/producer.properties
[KAFKA-556] - Change MessageSet.sizeInBytes to Int
[KAFKA-557] - Replica fetch thread doesn't need to recompute message id
[KAFKA-562] - Non-failure System Test Log Segment File Checksums mismatched
[KAFKA-563] - KafkaScheduler shutdown in ZookeeperConsumerConnector should check for config.autocommit
[KAFKA-567] - Replication Data Loss in Mirror Maker Bouncing testcase
[KAFKA-573] - System Test : Leader Failure Log Segment Checksum Mismatched When request-num-acks is 1
[KAFKA-575] - Partition.makeFollower() reads broker info from ZK
[KAFKA-576] - SimpleConsumer throws UnsupportedOperationException: empty.head
[KAFKA-577] - extend DumpLogSegments to verify consistency btw data and index
[KAFKA-578] - Leader finder thread in ConsumerFetcherManager needs to handle exceptions
[KAFKA-579] - remove connection timeout in SyncProducer
[KAFKA-580] - system test testcase_0122 under replication fails due to large # of data loss
[KAFKA-584] - produce/fetch remote time metric not set correctly when num.acks = 1
[KAFKA-586] - system test configs are broken
[KAFKA-591] - Add test cases to test log size retention and more
[KAFKA-592] - Register metrics beans at kafka server startup
[KAFKA-596] - LogSegment.firstAppendTime not reset after truncate to
[KAFKA-604] - Add missing metrics in 0.8
[KAFKA-608] - getTopicMetadata does not respect producer config settings
[KAFKA-612] - move shutting down of fetcher thread out of critical path
[KAFKA-613] - MigrationTool should disable shallow iteration in the 0.7 consumer
[KAFKA-614] - DumpLogSegment offset verification is incorrect for compressed messages
[KAFKA-618] - Deadlock between leader-finder-thread and consumer-fetcher-thread during broker failure
[KAFKA-622] - Create mbeans per client
[KAFKA-625] - Improve MessageAndMetadata to expose the partition
[KAFKA-634] - ConsoleProducer compresses messages and ignores the --compress flag
[KAFKA-646] - Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level
[KAFKA-648] - Use uniform convention for naming properties keys
[KAFKA-664] - Kafka server threads die due to OOME during long running test
[KAFKA-668] - Controlled shutdown admin tool should not require controller JMX url/port to be supplied
[KAFKA-669] - Irrecoverable error on leader while rolling to a new segment
[KAFKA-673] - Broker recovery check logic is reversed
[KAFKA-680] - ApiUtils#writeShortString uses String length instead of byte length
[KAFKA-681] - Unclean shutdown testing - truncateAndStartWithNewOffset is not invoked when it is expected to
[KAFKA-684] - ConsoleProducer does not have the queue-size option
[KAFKA-690] - TopicMetadataRequest throws exception when no topics are specified
[KAFKA-691] - Fault tolerance broken with replication factor 1
[KAFKA-692] - ConsoleConsumer outputs diagnostic message to stdout instead of stderr
[KAFKA-693] - Consumer rebalance fails if no leader available for a partition and stops all fetchers
[KAFKA-695] - Broker shuts down due to attempt to read a closed index file
[KAFKA-698] - broker may expose uncommitted data to a consumer
[KAFKA-701] - ConsoleProducer does not exit correctly and fix some config properties following KAFKA-648
[KAFKA-702] - Deadlock between request handler/processor threads
[KAFKA-708] - ISR becomes empty while marking a partition offline
[KAFKA-710] - Some arguments are always set to default in ProducerPerformance
[KAFKA-713] - Update Hadoop producer for Kafka 0.8 changes
[KAFKA-718] - kafka-run-class.sh should use reasonable gc settings
[KAFKA-726] - Add ReplicaFetcherThread name to mbean names
[KAFKA-732] - MirrorMaker with shallow.iterator.enable=true produces unreadble messages
[KAFKA-738] - correlationId is not set in FetchRequest in AbstractFetcherThread
[KAFKA-743] - PreferredReplicaLeaderElectionCommand has command line error
[KAFKA-748] - Append to index fails due to invalid offset
[KAFKA-750] - inconsistent index offset during broker startup
[KAFKA-751] - Fix windows build script - kafka-run-class.bat
[KAFKA-753] - Kafka broker shuts down while loading segments
[KAFKA-755] - standardizing json values stored in ZK
[KAFKA-756] - Processor thread blocks due to infinite loop during fetch response send
[KAFKA-757] - System Test Hard Failure cases : "Fatal error during KafkaServerStable startup" when hard-failed broker is re-started
[KAFKA-758] - startHighWaterMarksCheckPointThread is never called
[KAFKA-767] - Message Size check should be done after assigning the offsets
[KAFKA-768] - broker should exit if hitting exceptions durin startup
[KAFKA-769] - On startup, a brokers highwatermark for every topic partition gets reset to zero
[KAFKA-770] - KafkaConfig properties should be verified in the constructor
[KAFKA-772] - System Test Transient Failure on testcase_0122
[KAFKA-776] - Changing ZK format breaks some tools
[KAFKA-779] - Standardize Zk data structures for Re-assign partitions and Preferred replication election
[KAFKA-785] - Resolve bugs in PreferredReplicaLeaderElection admin tool
[KAFKA-786] - Use "withRequiredArg" while parsing jopt options in all tools
[KAFKA-793] - Include controllerId in all requests sent by controller
[KAFKA-798] - Use biased histograms instead of uniform histograms in KafkaMetricsGroup
[KAFKA-800] - inSyncReplica in Partition needs some tweaks
[KAFKA-801] - Fix MessagesInPerSec mbean to count uncompressed message rate
[KAFKA-804] - Incorrect index in the log of a follower
[KAFKA-807] - LineMessageReader doesn't correctly parse the key separator
[KAFKA-809] - Dependency on zkclient 0.1 (redundant) prevents building in IntelliJ
[KAFKA-811] - Fix clientId in migration tool
[KAFKA-813] - Minor cleanup in Controller
[KAFKA-825] - KafkaController.isActive() needs to be synchronized
[KAFKA-826] - Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[KAFKA-827] - improve list topic output format
[KAFKA-828] - Preferred Replica Election does not delete the admin path on controller failover
[KAFKA-830] - partition replica assignment map in the controller should be a Set
[KAFKA-832] - 0.8 Consumer prevents rebalance if consumer thread is blocked or slow
[KAFKA-840] - Controller tries to perform preferred replica election on failover before state machines have started up
[KAFKA-846] - AbstractFetcherThread should do shallow instead of deep iteration
[KAFKA-856] - Correlation id for OffsetFetch request (#2) always responds with 0
[KAFKA-861] - IndexOutOfBoundsException while fetching data from leader
[KAFKA-866] - Recover segment does shallow iteration to fix index causing inconsistencies
[KAFKA-871] - Rename ZkConfig properties
[KAFKA-872] - Socket server does not set send/recv buffer sizes
[KAFKA-879] - In system test, read the new leader from zookeeper instead of broker log on completion of become-leader state transition
[KAFKA-880] - NoLeaderPartitionSet should be cleared before leader finder thread is started up
[KAFKA-884] - Get java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J when stopping kafka brokers
[KAFKA-895] - Protocol documentation is not clear about requiredAcks = 0.
[KAFKA-899] - LeaderNotAvailableException the first time a new message for a partition is processed.
[KAFKA-900] - ClosedByInterruptException when high-level consumer shutdown normally
[KAFKA-903] - [0.8.0 - windows] FATAL - [highwatermark-checkpoint-thread1] (Logging.scala:109) - Attempt to swap the new high watermark file with the old one failed
[KAFKA-905] - Logs can have same offsets causing recovery failure
[KAFKA-907] - controller needs to close socket channel to brokers on exception
[KAFKA-914] - Deadlock between initial rebalance and watcher-triggered rebalances
[KAFKA-916] - Deadlock between fetcher shutdown and handling partitions with error
[KAFKA-919] - Disabling of auto commit is ignored during consumer group rebalancing
[KAFKA-920] - zkclient jar 0.2.0 is not compatible with 0.1.0
[KAFKA-921] - Expose max lag mbean for consumers and replica fetchers
[KAFKA-927] - Integrate controlled shutdown into kafka shutdown hook
[KAFKA-937] - ConsumerFetcherThread can deadlock
[KAFKA-938] - High CPU usage when more or less idle
[KAFKA-939] - ./sbt publish-local fails due to invalid javac flags passed to javadoc
[KAFKA-940] - Scala match error in javaapi.Implicits
[KAFKA-941] - Add Apache 2.0 license to missing code source files
[KAFKA-942] - the version of the jar should be 0.8.0-beta1 not 0.8.0-SNAPSHOT
[KAFKA-944] - the pom output from publish and publish-local is not accurate
[KAFKA-946] - Kafka Hadoop Consumer fails when verifying message checksum
[KAFKA-947] - isr-expiration-thread may block LeaderAndIsr request for a relatively long period
[KAFKA-950] - bytesSinceLastIndexEntry needs to be reset after log segment is truncated
[KAFKA-951] - Leader election rate may be reported on a non-controller
[KAFKA-953] - Remove release-zip from README we are not releasing with it
[KAFKA-954] - tidy up README file for better general availability
[KAFKA-955] - After a leader change, messages sent with ack=0 are lost
[KAFKA-956] - High-level consumer fails to check topic metadata response for errors
[KAFKA-959] - DefaultEventHandler can send more produce requests than necesary
[KAFKA-963] - when publishing to maven repository central missing signature on everything
[KAFKA-964] - Default hadoop-producer configs to request.required.acks = 1
[KAFKA-968] - Typographical Errors in Output
[KAFKA-969] - Need to prevent failure of rebalance when there are no brokers available when consumer comes up
[KAFKA-974] - can't use public release maven repo because of failure of downloaded dependency
[KAFKA-978] - kafka pom file has 2 entries for zookeeper (one with exclusion, one without)
[KAFKA-987] - Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit
[KAFKA-989] - Race condition shutting down high-level consumer results in spinning background thread
[KAFKA-991] - Reduce the queue size in hadoop producer
[KAFKA-994] - High level consumer doesn't throw an exception when the message it is trying to fetch exceeds the configured fetch size
[KAFKA-1003] - ConsumerFetcherManager should pass clientId as metricsPrefix to AbstractFetcherManager
[KAFKA-1008] - Unmap before resizing
[KAFKA-1010] - Concurrency issue in getCluster() causes rebalance failure and dead consumer
[KAFKA-1017] - High number of open file handles in 0.8 producer
[KAFKA-1018] - tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[KAFKA-1029] - Zookeeper leader election stuck in ephemeral node retry loop
[KAFKA-1030] - Addition of partitions requires bouncing all the consumers of that topic
[KAFKA-1031] - Little modification to the stop script to be able to kill the proper process
[KAFKA-1035] - Add message-send-max-retries and retry-backoff-ms options to console producer
[KAFKA-1038] - fetch response should use empty messageset instead of null when handling errors
[KAFKA-1068] - OfflinePartitionCount metrics may be incorrect after the controller failover
[KAFKA-1069] - MBean kafka.cluster.Partition report wrong UnderReplicated status
[KAFKA-1071] - The random partition selected in DefaultEventHandler is not random across producer instances
[KAFKA-1073] - CheckReassignmentStatus is broken
[KAFKA-1075] - Consumer will not rebalance upon topic partition change
[KAFKA-1076] - system tests in 0.8 are broken due to wrong log4j config
[KAFKA-1078] - Update System Test to handle controller data returned by ZK
[KAFKA-1116] - Need to upgrade sbt-assembly to compile on scala 2.10.2
[KAFKA-1126] - Remove the DISCLAIMER it is left over from incubation
[KAFKA-1131] - copy some more files into the release tar and zip that are needed/desired
[KAFKA-1133] - LICENSE and NOTICE files need to get into  META-INF when jars are built before they're signed for publishing to maven

Sub-task
[KAFKA-176] - Fix existing perf tools
[KAFKA-237] - create/delete ZK path for a topic in an admin tool
[KAFKA-239] - Wire existing producer and consumer to use the new ZK data structure
[KAFKA-240] - implement new producer and consumer request format
[KAFKA-329] - Remove the watches/broker for new topics and partitions and change create topic admin API to send start replica state change to all brokers
[KAFKA-335] - Implement an embedded controller
[KAFKA-336] - add an admin RPC to communicate state changes between the controller and the broker
[KAFKA-337] - upgrade ZKClient to allow conditional updates in ZK
[KAFKA-338] - controller failover
[KAFKA-339] - using MultiFetch in the follower
[KAFKA-342] - revisit the broker startup procedure according to V3 design
[KAFKA-343] - revisit the become leader and become follower state change operations using V3 design
[KAFKA-344] - migration tool from 0.7 to 0.8
[KAFKA-356] - Create a generic Kafka thread class that includes basic boiler plate code of instantiating and shutting down threads cleanly
[KAFKA-362] - ZookeeperConsumerConnector needs to connect to new leader after leadership change
[KAFKA-369] - remove ZK dependency on producer
[KAFKA-458] - remove errorcode from ByteBufferMessageSet
[KAFKA-482] - Make producer to run for the entire duration of the System Test
[KAFKA-488] - Port Mirroring System Test to this python system test framework
[KAFKA-492] - Sometimes the python system test framework doesn't terminate all running processes
[KAFKA-494] - Relative paths should be used for svg URLs in dashboards html
[KAFKA-502] - Simplify setup / initialization in replication_basic_test.py
[KAFKA-503] - Support "testcase_to_run" or "testcase_to_skip"
[KAFKA-507] - Shut down ZK last to avoid hanging brokers running processes
[KAFKA-513] - Add state change log to Kafka brokers
[KAFKA-571] - Add more test cases to System Test
[KAFKA-731] - ~/ivy2/cache should be a variable in the kafka-run-class bash script
[KAFKA-780] - Reassign partitions tool produces NPE in shutdown handler
[KAFKA-814] - Controller should not throw exception when a preferred replica is already the leader for a partition
[KAFKA-843] - Re-add the release-zip sbt target
Bug
[KAFKA-15] - SBT release-zip target doesn't include bin and config directories anymore
[KAFKA-42] - Support rebalancing the partitions with replication
[KAFKA-43] - Rebalance to preferred broke with intra-cluster replication support
[KAFKA-46] - Commit thread, ReplicaFetcherThread for intra-cluster replication
[KAFKA-49] - Add acknowledgement to the produce request.
[KAFKA-81] - wrong path in bin/kafka-run-class.sh
[KAFKA-97] - SocketServer.scala refers to Handler-specific variables
[KAFKA-171] - Kafka producer should do a single write to send message sets
[KAFKA-192] - CompressionUtilTest does not run and fails when it does
[KAFKA-215] - Improve system tests for the mirroring code
[KAFKA-229] - SimpleConsumer is not logging exceptions correctly so detailed stack trace is not coming in the logs
[KAFKA-259] - Give better error message when trying to run shell scripts without having built/downloaded the jars yet
[KAFKA-295] - Bug in async producer DefaultEventHandler retry logic
[KAFKA-305] - SyncProducer does not correctly timeout
[KAFKA-306] - broker failure system test broken on replication branch
[KAFKA-351] - Refactor some new components introduced for replication
[KAFKA-352] - Throw exception to client if it makes a produce/consume request to a Kafka broker for a topic that hasn't been created
[KAFKA-367] - StringEncoder/StringDecoder use platform default character set
[KAFKA-370] - Exception "java.util.NoSuchElementException: None.get" appears inconsistently in Mirror Maker log.
[KAFKA-371] - Creating topic of empty string puts broker in a bad state
[KAFKA-376] - expose different data to fetch requests from the follower replicas and consumer clients
[KAFKA-379] - TopicCount.constructTopicCount isn't thread-safe
[KAFKA-382] - Write ordering guarantee violated
[KAFKA-385] - RequestPurgatory enhancements - expire/checkSatisfy issue; add jmx beans
[KAFKA-386] - Race condition in accessing ISR
[KAFKA-391] - Producer request and response classes should use maps
[KAFKA-396] - Mirroring system test fails on 0.8
[KAFKA-412] - deal with empty TopicData list in producer and fetch request
[KAFKA-413] - single_host_multi_brokers system test fails on laptop
[KAFKA-415] - Controller throws NoSuchElementException while marking a broker failed
[KAFKA-416] - Controller tests throw several zookeeper errors
[KAFKA-418] - NullPointerException in ConsumerFetcherManager
[KAFKA-420] - maintain HW correctly with only 1 replica in ISR
[KAFKA-422] - LazyInitProducerTest has transient test failure
[KAFKA-424] - Remove invalid mirroring arguments from kafka-server-start.sh
[KAFKA-425] - Wrong class name in performance test scripts
[KAFKA-427] - LogRecoverTest.testHWCheckpointWithFailuresSingleLogSegment has transient failure
[KAFKA-428] - need to update leaderAndISR path in ZK conditionally in ReplicaManager
[KAFKA-431] - LogCorruptionTest.testMessageSizeTooLarge fails occasionally
[KAFKA-433] - ensurePartitionLeaderOnThisBroker should not access ZK
[KAFKA-434] - IllegalMonitorStateException in ReplicaManager.makerFollower
[KAFKA-452] - follower replica may need to backoff the fetching if leader is not ready yet
[KAFKA-453] - follower replica may need to backoff the fetching if leader is not ready yet
[KAFKA-456] - ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation
[KAFKA-459] - KafkaController.RequestSendThread can throw exception on broker socket
[KAFKA-460] - ControllerChannelManager needs synchronization btw shutdown and add/remove broker
[KAFKA-461] - remove support for format for magic byte 0 in 0.8
[KAFKA-463] - log.truncateTo needs to handle targetOffset smaller than the lowest offset in the log
[KAFKA-464] - KafkaController NPE in SessionExpireListener
[KAFKA-466] - Controller tests throw IllegalStateException
[KAFKA-467] - Controller based leader election failed ERROR messages in LazyInitProducerTest
[KAFKA-468] - String#getBytes is platform dependent
[KAFKA-470] - transient unit test failure in RequestPurgatoryTest
[KAFKA-471] - Transient failure in ProducerTest
[KAFKA-473] - Use getMetadata Api in ZookeeperConsumerConnector
[KAFKA-474] - support changing host/port of a broker
[KAFKA-481] - Require values in Utils.getTopic* methods to be positive
[KAFKA-490] - Check max message size on server instead of producer
[KAFKA-491] - KafkaRequestHandler needs to handle exceptions
[KAFKA-495] - Handle topic names with "/" on Kafka server
[KAFKA-497] - recover consumer during unclean leadership change
[KAFKA-499] - Refactor controller state machine
[KAFKA-500] - javaapi support for getTopoicMetaData
[KAFKA-501] - getOfffset Api needs to return different latest offset to regular and follower consumers
[KAFKA-506] - Store logical offset in log
[KAFKA-508] - split out partiondata from fetchresponse and producerrequest
[KAFKA-509] - server should shut down on encountering invalid highwatermark file
[KAFKA-510] - broker needs to know the replication factor per partition
[KAFKA-511] - offset returned in Producer response may not be correct
[KAFKA-512] - Remove checksum from ByteBufferMessageSet.iterator
[KAFKA-514] - Replication with Leader Failure Test: Log segment files checksum mismatch
[KAFKA-516] - Consider catching all exceptions in ShutdownableThread
[KAFKA-525] - newly created partitions are not added to ReplicaStateMachine
[KAFKA-528] - IndexOutOfBoundsException thrown by kafka.consumer.ConsumerFetcherThread
[KAFKA-531] - kafka.server.ReplicaManager: java.nio.channels.NonWritableChannelException
[KAFKA-537] - expose clientId and correlationId in ConsumerConfig
[KAFKA-539] - Replica.hw should be initialized to the smaller of checkedpointed HW and log end offset
[KAFKA-540] - log.append() should halt on IOException
[KAFKA-553] - confusing reference to zk.connect in config/producer.properties
[KAFKA-556] - Change MessageSet.sizeInBytes to Int
[KAFKA-557] - Replica fetch thread doesn't need to recompute message id
[KAFKA-562] - Non-failure System Test Log Segment File Checksums mismatched
[KAFKA-563] - KafkaScheduler shutdown in ZookeeperConsumerConnector should check for config.autocommit
[KAFKA-567] - Replication Data Loss in Mirror Maker Bouncing testcase
[KAFKA-573] - System Test : Leader Failure Log Segment Checksum Mismatched When request-num-acks is 1
[KAFKA-575] - Partition.makeFollower() reads broker info from ZK
[KAFKA-576] - SimpleConsumer throws UnsupportedOperationException: empty.head
[KAFKA-577] - extend DumpLogSegments to verify consistency btw data and index
[KAFKA-578] - Leader finder thread in ConsumerFetcherManager needs to handle exceptions
[KAFKA-579] - remove connection timeout in SyncProducer
[KAFKA-580] - system test testcase_0122 under replication fails due to large # of data loss
[KAFKA-584] - produce/fetch remote time metric not set correctly when num.acks = 1
[KAFKA-586] - system test configs are broken
[KAFKA-591] - Add test cases to test log size retention and more
[KAFKA-592] - Register metrics beans at kafka server startup
[KAFKA-596] - LogSegment.firstAppendTime not reset after truncate to
[KAFKA-604] - Add missing metrics in 0.8
[KAFKA-608] - getTopicMetadata does not respect producer config settings
[KAFKA-612] - move shutting down of fetcher thread out of critical path
[KAFKA-613] - MigrationTool should disable shallow iteration in the 0.7 consumer
[KAFKA-614] - DumpLogSegment offset verification is incorrect for compressed messages
[KAFKA-618] - Deadlock between leader-finder-thread and consumer-fetcher-thread during broker failure
[KAFKA-622] - Create mbeans per client
[KAFKA-625] - Improve MessageAndMetadata to expose the partition
[KAFKA-634] - ConsoleProducer compresses messages and ignores the --compress flag
[KAFKA-646] - Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level
[KAFKA-648] - Use uniform convention for naming properties keys
[KAFKA-664] - Kafka server threads die due to OOME during long running test
[KAFKA-668] - Controlled shutdown admin tool should not require controller JMX url/port to be supplied
[KAFKA-669] - Irrecoverable error on leader while rolling to a new segment
[KAFKA-673] - Broker recovery check logic is reversed
[KAFKA-680] - ApiUtils#writeShortString uses String length instead of byte length
[KAFKA-681] - Unclean shutdown testing - truncateAndStartWithNewOffset is not invoked when it is expected to
[KAFKA-684] - ConsoleProducer does not have the queue-size option
[KAFKA-690] - TopicMetadataRequest throws exception when no topics are specified
[KAFKA-691] - Fault tolerance broken with replication factor 1
[KAFKA-692] - ConsoleConsumer outputs diagnostic message to stdout instead of stderr
[KAFKA-693] - Consumer rebalance fails if no leader available for a partition and stops all fetchers
[KAFKA-695] - Broker shuts down due to attempt to read a closed index file
[KAFKA-698] - broker may expose uncommitted data to a consumer
[KAFKA-701] - ConsoleProducer does not exit correctly and fix some config properties following KAFKA-648
[KAFKA-702] - Deadlock between request handler/processor threads
[KAFKA-708] - ISR becomes empty while marking a partition offline
[KAFKA-710] - Some arguments are always set to default in ProducerPerformance
[KAFKA-713] - Update Hadoop producer for Kafka 0.8 changes
[KAFKA-726] - Add ReplicaFetcherThread name to mbean names
[KAFKA-732] - MirrorMaker with shallow.iterator.enable=true produces unreadble messages
[KAFKA-738] - correlationId is not set in FetchRequest in AbstractFetcherThread
[KAFKA-743] - PreferredReplicaLeaderElectionCommand has command line error
[KAFKA-748] - Append to index fails due to invalid offset
[KAFKA-750] - inconsistent index offset during broker startup
[KAFKA-751] - Fix windows build script - kafka-run-class.bat
[KAFKA-753] - Kafka broker shuts down while loading segments
[KAFKA-755] - standardizing json values stored in ZK
[KAFKA-756] - Processor thread blocks due to infinite loop during fetch response send
[KAFKA-757] - System Test Hard Failure cases : "Fatal error during KafkaServerStable startup" when hard-failed broker is re-started
[KAFKA-758] - startHighWaterMarksCheckPointThread is never called
[KAFKA-767] - Message Size check should be done after assigning the offsets
[KAFKA-768] - broker should exit if hitting exceptions durin startup
[KAFKA-769] - On startup, a brokers highwatermark for every topic partition gets reset to zero
[KAFKA-770] - KafkaConfig properties should be verified in the constructor
[KAFKA-772] - System Test Transient Failure on testcase_0122
[KAFKA-776] - Changing ZK format breaks some tools
[KAFKA-779] - Standardize Zk data structures for Re-assign partitions and Preferred replication election
[KAFKA-785] - Resolve bugs in PreferredReplicaLeaderElection admin tool
[KAFKA-786] - Use "withRequiredArg" while parsing jopt options in all tools
[KAFKA-793] - Include controllerId in all requests sent by controller
[KAFKA-798] - Use biased histograms instead of uniform histograms in KafkaMetricsGroup
[KAFKA-800] - inSyncReplica in Partition needs some tweaks
[KAFKA-801] - Fix MessagesInPerSec mbean to count uncompressed message rate
[KAFKA-804] - Incorrect index in the log of a follower
[KAFKA-807] - LineMessageReader doesn't correctly parse the key separator
[KAFKA-809] - Dependency on zkclient 0.1 (redundant) prevents building in IntelliJ
[KAFKA-811] - Fix clientId in migration tool
[KAFKA-813] - Minor cleanup in Controller
[KAFKA-825] - KafkaController.isActive() needs to be synchronized
[KAFKA-826] - Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[KAFKA-827] - improve list topic output format
[KAFKA-828] - Preferred Replica Election does not delete the admin path on controller failover
[KAFKA-830] - partition replica assignment map in the controller should be a Set
[KAFKA-832] - 0.8 Consumer prevents rebalance if consumer thread is blocked or slow
[KAFKA-840] - Controller tries to perform preferred replica election on failover before state machines have started up
[KAFKA-846] - AbstractFetcherThread should do shallow instead of deep iteration
[KAFKA-856] - Correlation id for OffsetFetch request (#2) always responds with 0
[KAFKA-861] - IndexOutOfBoundsException while fetching data from leader
[KAFKA-866] - Recover segment does shallow iteration to fix index causing inconsistencies
[KAFKA-871] - Rename ZkConfig properties
[KAFKA-872] - Socket server does not set send/recv buffer sizes
[KAFKA-880] - NoLeaderPartitionSet should be cleared before leader finder thread is started up
[KAFKA-884] - Get java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J when stopping kafka brokers
[KAFKA-895] - Protocol documentation is not clear about requiredAcks = 0.
[KAFKA-899] - LeaderNotAvailableException the first time a new message for a partition is processed.
[KAFKA-900] - ClosedByInterruptException when high-level consumer shutdown normally
[KAFKA-903] - [0.8.0 - windows] FATAL - [highwatermark-checkpoint-thread1] (Logging.scala:109) - Attempt to swap the new high watermark file with the old one failed
[KAFKA-905] - Logs can have same offsets causing recovery failure
[KAFKA-907] - controller needs to close socket channel to brokers on exception
[KAFKA-914] - Deadlock between initial rebalance and watcher-triggered rebalances
[KAFKA-916] - Deadlock between fetcher shutdown and handling partitions with error
[KAFKA-919] - Disabling of auto commit is ignored during consumer group rebalancing
[KAFKA-920] - zkclient jar 0.2.0 is not compatible with 0.1.0
[KAFKA-921] - Expose max lag mbean for consumers and replica fetchers
[KAFKA-927] - Integrate controlled shutdown into kafka shutdown hook
[KAFKA-937] - ConsumerFetcherThread can deadlock
[KAFKA-938] - High CPU usage when more or less idle
[KAFKA-940] - Scala match error in javaapi.Implicits
[KAFKA-941] - Add Apache 2.0 license to missing code source files
[KAFKA-942] - the version of the jar should be 0.8.0-beta1 not 0.8.0-SNAPSHOT
[KAFKA-944] - the pom output from publish and publish-local is not accurate
Improvement
[KAFKA-77] - Implement "group commit" for kafka logs
[KAFKA-100] - ProducerShell should use high-level producer instead of SyncProducer
[KAFKA-133] - Publish kafka jar to a public maven repository
[KAFKA-134] - Upgrade Kafka to sbt 0.11.3
[KAFKA-139] - cross-compile multiple Scala versions and upgrade to SBT 0.12.1
[KAFKA-155] - Support graceful Decommissioning of Broker
[KAFKA-165] - Add helper script for zkCli.sh
[KAFKA-181] - Log errors for unrecognized config options
[KAFKA-187] - Add Snappy Compression as a Codec and refactor CompressionUtil and option on startup to select what the default codec
[KAFKA-193] - use by name parameter helper for logging and trait to include lazy logging and refactor code to use the new LogHelper
[KAFKA-246] - log configuration values used
[KAFKA-253] - Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster
[KAFKA-258] - Remove broker.id from the broker.list config in the Producer
[KAFKA-267] - Enhance ProducerPerformance to generate unique random Long value for payload
[KAFKA-271] - Modify new FetchResponse object to remove the initial offset field
[KAFKA-281] - support multiple root log directories
[KAFKA-285] - Increase maximum value of log.retention.size
[KAFKA-296] - Update Go Client to new version of Go
[KAFKA-311] - Allow KafkaLog4jAppender to take in a configurable producer.type
[KAFKA-314] - Go Client Multi-produce
[KAFKA-323] - Add the ability to use the async producer in the Log4j appender
[KAFKA-324] - enforce broker.id to be a non-negative integer
[KAFKA-325] - revisit broker config in 0.8
[KAFKA-349] - Create individual "Response" types for each kind of request and wrap them with "BoundedByteBufferSend", remove "xxResponseSend" types for all requests except "FetchRequest"
[KAFKA-365] - change copyright in NOTICE to 2012
[KAFKA-366] - add jmx beans in broker to track # bytes in consumer
[KAFKA-368] - use the pig core jar from maven instead of distributing it
[KAFKA-393] - Add constructor for message which takes both byte array offset and length
[KAFKA-408] - ProducerPerformance does not work with all producer config options
[KAFKA-437] - Unused var statement in ZookeeperConsumerConnectorTest
[KAFKA-439] - @returns was used in scala doc when it should have been @return
[KAFKA-505] - Remove errorcode from TopicMetaDataResponse
[KAFKA-548] - remove partition from ProducerRequestPartitionData and FetchResponsePartitionData
[KAFKA-581] - provides windows batch script for starting Kafka/Zookeeper
[KAFKA-632] - ProducerRequest should take ByteBufferMessageSet instead of MessageSet
[KAFKA-638] - remove ProducerShell
[KAFKA-667] - Rename .highwatermark file
[KAFKA-675] - Only bind to the interface declared in the 'hostname' config property
[KAFKA-699] - Disallow clients to set replicaId in FetchRequest
[KAFKA-733] - Fat jar option for build, or override for ivy cache location
[KAFKA-762] - Improve second replica assignment
[KAFKA-763] - Add an option to replica from the largest offset during unclean leader election
[KAFKA-812] - Support deep iteration in DumpLogSegments tool
[KAFKA-850] - add an option to show under replicated partitions in list topic command
[KAFKA-931] - make zookeeper.connect a required property
New Feature
[KAFKA-50] - kafka intra-cluster replication support
[KAFKA-188] - Support multiple data directories
[KAFKA-202] - Make the request processing in kafka asynchonous
[KAFKA-203] - Improve Kafka internal metrics
[KAFKA-235] - Add a 'log.file.age' configuration parameter to force rotation of log files after they've reached a certain age
[KAFKA-429] - Expose JMX operation to set logger level dynamically
[KAFKA-475] - Time based log segment rollout
[KAFKA-545] - Add a Performance Suite for the Log subsystem
[KAFKA-546] - Fix commit() in zk consumer for compressed messages
Task
[KAFKA-93] - Change code header to follow standard ASF source header
[KAFKA-317] - Add support for new wire protocol to Go client
[KAFKA-341] - Create a new single host system test to validate all replicas on 0.8 branch
[KAFKA-348] - rebase 0.8 branch from trunk
[KAFKA-380] - Enhance single_host_multi_brokers test with failure to trigger leader re-election in replication
[KAFKA-440] - Create a regression test framework for distributed environment testing
[KAFKA-526] - System Test should remove the top level data log directory
[KAFKA-594] - Update System Test due to new argument "--sync" in ProducerPerformance
[KAFKA-605] - System Test - Log Retention Cases should wait longer before getting the common starting offset in replica log segments
[KAFKA-688] - System Test - Update all testcase_xxxx_properties.json for properties keys uniform naming convention
[KAFKA-737] - System Test : Disable shallow.iterator in Mirror Maker test cases to make compression work correctly
[KAFKA-791] - Fix validation bugs in System Test
[KAFKA-792] - Update multiple attributes in testcase_xxxx_properties.json
[KAFKA-819] - System Test : Add validation of log segment index to offset

Sub-task
[KAFKA-176] - Fix existing perf tools
[KAFKA-237] - create/delete ZK path for a topic in an admin tool
[KAFKA-239] - Wire existing producer and consumer to use the new ZK data structure
[KAFKA-240] - implement new producer and consumer request format
[KAFKA-329] - Remove the watches/broker for new topics and partitions and change create topic admin API to send start replica state change to all brokers
[KAFKA-335] - Implement an embedded controller
[KAFKA-336] - add an admin RPC to communicate state changes between the controller and the broker
[KAFKA-337] - upgrade ZKClient to allow conditional updates in ZK
[KAFKA-338] - controller failover
[KAFKA-339] - using MultiFetch in the follower
[KAFKA-342] - revisit the broker startup procedure according to V3 design
[KAFKA-343] - revisit the become leader and become follower state change operations using V3 design
[KAFKA-344] - migration tool from 0.7 to 0.8
[KAFKA-356] - Create a generic Kafka thread class that includes basic boiler plate code of instantiating and shutting down threads cleanly
[KAFKA-362] - ZookeeperConsumerConnector needs to connect to new leader after leadership change
[KAFKA-369] - remove ZK dependency on producer
[KAFKA-458] - remove errorcode from ByteBufferMessageSet
[KAFKA-482] - Make producer to run for the entire duration of the System Test
[KAFKA-488] - Port Mirroring System Test to this python system test framework
[KAFKA-492] - Sometimes the python system test framework doesn't terminate all running processes
[KAFKA-494] - Relative paths should be used for svg URLs in dashboards html
[KAFKA-502] - Simplify setup / initialization in replication_basic_test.py
[KAFKA-503] - Support "testcase_to_run" or "testcase_to_skip"
[KAFKA-507] - Shut down ZK last to avoid hanging brokers running processes
[KAFKA-513] - Add state change log to Kafka brokers
[KAFKA-571] - Add more test cases to System Test
[KAFKA-731] - ~/ivy2/cache should be a variable in the kafka-run-class bash script
[KAFKA-780] - Reassign partitions tool produces NPE in shutdown handler
[KAFKA-814] - Controller should not throw exception when a preferred replica is already the leader for a partition
[KAFKA-843] - Re-add the release-zip sbt target
Bug
[KAFKA-42] - Support rebalancing the partitions with replication
[KAFKA-43] - Rebalance to preferred broke with intra-cluster replication support
[KAFKA-46] - Commit thread, ReplicaFetcherThread for intra-cluster replication
[KAFKA-49] - Add acknowledgement to the produce request.
[KAFKA-97] - SocketServer.scala refers to Handler-specific variables
[KAFKA-171] - Kafka producer should do a single write to send message sets
[KAFKA-192] - CompressionUtilTest does not run and fails when it does
[KAFKA-215] - Improve system tests for the mirroring code
[KAFKA-229] - SimpleConsumer is not logging exceptions correctly so detailed stack trace is not coming in the logs
[KAFKA-259] - Give better error message when trying to run shell scripts without having built/downloaded the jars yet
[KAFKA-295] - Bug in async producer DefaultEventHandler retry logic
[KAFKA-305] - SyncProducer does not correctly timeout
[KAFKA-306] - broker failure system test broken on replication branch
[KAFKA-351] - Refactor some new components introduced for replication
[KAFKA-352] - Throw exception to client if it makes a produce/consume request to a Kafka broker for a topic that hasn't been created
[KAFKA-367] - StringEncoder/StringDecoder use platform default character set
[KAFKA-370] - Exception "java.util.NoSuchElementException: None.get" appears inconsistently in Mirror Maker log.
[KAFKA-371] - Creating topic of empty string puts broker in a bad state
[KAFKA-376] - expose different data to fetch requests from the follower replicas and consumer clients
[KAFKA-379] - TopicCount.constructTopicCount isn't thread-safe
[KAFKA-382] - Write ordering guarantee violated
[KAFKA-385] - RequestPurgatory enhancements - expire/checkSatisfy issue; add jmx beans
[KAFKA-386] - Race condition in accessing ISR
[KAFKA-391] - Producer request and response classes should use maps
[KAFKA-412] - deal with empty TopicData list in producer and fetch request
[KAFKA-413] - single_host_multi_brokers system test fails on laptop
[KAFKA-415] - Controller throws NoSuchElementException while marking a broker failed
[KAFKA-416] - Controller tests throw several zookeeper errors
[KAFKA-418] - NullPointerException in ConsumerFetcherManager
[KAFKA-420] - maintain HW correctly with only 1 replica in ISR
[KAFKA-422] - LazyInitProducerTest has transient test failure
[KAFKA-424] - Remove invalid mirroring arguments from kafka-server-start.sh
[KAFKA-425] - Wrong class name in performance test scripts
[KAFKA-427] - LogRecoverTest.testHWCheckpointWithFailuresSingleLogSegment has transient failure
[KAFKA-428] - need to update leaderAndISR path in ZK conditionally in ReplicaManager
[KAFKA-431] - LogCorruptionTest.testMessageSizeTooLarge fails occasionally
[KAFKA-433] - ensurePartitionLeaderOnThisBroker should not access ZK
[KAFKA-434] - IllegalMonitorStateException in ReplicaManager.makerFollower
[KAFKA-452] - follower replica may need to backoff the fetching if leader is not ready yet
[KAFKA-453] - follower replica may need to backoff the fetching if leader is not ready yet
[KAFKA-456] - ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation
[KAFKA-459] - KafkaController.RequestSendThread can throw exception on broker socket
[KAFKA-460] - ControllerChannelManager needs synchronization btw shutdown and add/remove broker
[KAFKA-461] - remove support for format for magic byte 0 in 0.8
[KAFKA-463] - log.truncateTo needs to handle targetOffset smaller than the lowest offset in the log
[KAFKA-464] - KafkaController NPE in SessionExpireListener
[KAFKA-466] - Controller tests throw IllegalStateException
[KAFKA-467] - Controller based leader election failed ERROR messages in LazyInitProducerTest
[KAFKA-468] - String#getBytes is platform dependent
[KAFKA-470] - transient unit test failure in RequestPurgatoryTest
[KAFKA-471] - Transient failure in ProducerTest
[KAFKA-473] - Use getMetadata Api in ZookeeperConsumerConnector
[KAFKA-474] - support changing host/port of a broker
[KAFKA-481] - Require values in Utils.getTopic* methods to be positive
[KAFKA-490] - Check max message size on server instead of producer
[KAFKA-491] - KafkaRequestHandler needs to handle exceptions
[KAFKA-495] - Handle topic names with "/" on Kafka server
[KAFKA-497] - recover consumer during unclean leadership change
[KAFKA-499] - Refactor controller state machine
[KAFKA-500] - javaapi support for getTopoicMetaData
[KAFKA-501] - getOfffset Api needs to return different latest offset to regular and follower consumers
[KAFKA-506] - Store logical offset in log
[KAFKA-508] - split out partiondata from fetchresponse and producerrequest
[KAFKA-509] - server should shut down on encountering invalid highwatermark file
[KAFKA-510] - broker needs to know the replication factor per partition
[KAFKA-511] - offset returned in Producer response may not be correct
[KAFKA-512] - Remove checksum from ByteBufferMessageSet.iterator
[KAFKA-514] - Replication with Leader Failure Test: Log segment files checksum mismatch
[KAFKA-516] - Consider catching all exceptions in ShutdownableThread
[KAFKA-525] - newly created partitions are not added to ReplicaStateMachine
[KAFKA-528] - IndexOutOfBoundsException thrown by kafka.consumer.ConsumerFetcherThread
[KAFKA-531] - kafka.server.ReplicaManager: java.nio.channels.NonWritableChannelException
[KAFKA-537] - expose clientId and correlationId in ConsumerConfig
[KAFKA-539] - Replica.hw should be initialized to the smaller of checkedpointed HW and log end offset
[KAFKA-540] - log.append() should halt on IOException
[KAFKA-553] - confusing reference to zk.connect in config/producer.properties
[KAFKA-556] - Change MessageSet.sizeInBytes to Int
[KAFKA-557] - Replica fetch thread doesn't need to recompute message id
[KAFKA-562] - Non-failure System Test Log Segment File Checksums mismatched
[KAFKA-563] - KafkaScheduler shutdown in ZookeeperConsumerConnector should check for config.autocommit
[KAFKA-567] - Replication Data Loss in Mirror Maker Bouncing testcase
[KAFKA-573] - System Test : Leader Failure Log Segment Checksum Mismatched When request-num-acks is 1
[KAFKA-575] - Partition.makeFollower() reads broker info from ZK
[KAFKA-576] - SimpleConsumer throws UnsupportedOperationException: empty.head
[KAFKA-577] - extend DumpLogSegments to verify consistency btw data and index
[KAFKA-578] - Leader finder thread in ConsumerFetcherManager needs to handle exceptions
[KAFKA-579] - remove connection timeout in SyncProducer
[KAFKA-580] - system test testcase_0122 under replication fails due to large # of data loss
[KAFKA-584] - produce/fetch remote time metric not set correctly when num.acks = 1
[KAFKA-586] - system test configs are broken
[KAFKA-591] - Add test cases to test log size retention and more
[KAFKA-592] - Register metrics beans at kafka server startup
[KAFKA-596] - LogSegment.firstAppendTime not reset after truncate to
[KAFKA-604] - Add missing metrics in 0.8
[KAFKA-608] - getTopicMetadata does not respect producer config settings
[KAFKA-612] - move shutting down of fetcher thread out of critical path
[KAFKA-613] - MigrationTool should disable shallow iteration in the 0.7 consumer
[KAFKA-614] - DumpLogSegment offset verification is incorrect for compressed messages
[KAFKA-618] - Deadlock between leader-finder-thread and consumer-fetcher-thread during broker failure
[KAFKA-622] - Create mbeans per client
[KAFKA-625] - Improve MessageAndMetadata to expose the partition
[KAFKA-634] - ConsoleProducer compresses messages and ignores the --compress flag
[KAFKA-646] - Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level
[KAFKA-648] - Use uniform convention for naming properties keys
[KAFKA-664] - Kafka server threads die due to OOME during long running test
[KAFKA-668] - Controlled shutdown admin tool should not require controller JMX url/port to be supplied
[KAFKA-669] - Irrecoverable error on leader while rolling to a new segment
[KAFKA-673] - Broker recovery check logic is reversed
[KAFKA-680] - ApiUtils#writeShortString uses String length instead of byte length
[KAFKA-681] - Unclean shutdown testing - truncateAndStartWithNewOffset is not invoked when it is expected to
[KAFKA-684] - ConsoleProducer does not have the queue-size option
[KAFKA-690] - TopicMetadataRequest throws exception when no topics are specified
[KAFKA-691] - Fault tolerance broken with replication factor 1
[KAFKA-692] - ConsoleConsumer outputs diagnostic message to stdout instead of stderr
[KAFKA-693] - Consumer rebalance fails if no leader available for a partition and stops all fetchers
[KAFKA-695] - Broker shuts down due to attempt to read a closed index file
[KAFKA-698] - broker may expose uncommitted data to a consumer
[KAFKA-701] - ConsoleProducer does not exit correctly and fix some config properties following KAFKA-648
[KAFKA-702] - Deadlock between request handler/processor threads
[KAFKA-708] - ISR becomes empty while marking a partition offline
[KAFKA-710] - Some arguments are always set to default in ProducerPerformance
[KAFKA-713] - Update Hadoop producer for Kafka 0.8 changes
[KAFKA-726] - Add ReplicaFetcherThread name to mbean names
[KAFKA-732] - MirrorMaker with shallow.iterator.enable=true produces unreadble messages
[KAFKA-738] - correlationId is not set in FetchRequest in AbstractFetcherThread
[KAFKA-743] - PreferredReplicaLeaderElectionCommand has command line error
[KAFKA-748] - Append to index fails due to invalid offset
[KAFKA-750] - inconsistent index offset during broker startup
[KAFKA-751] - Fix windows build script - kafka-run-class.bat
[KAFKA-753] - Kafka broker shuts down while loading segments
[KAFKA-755] - standardizing json values stored in ZK
[KAFKA-756] - Processor thread blocks due to infinite loop during fetch response send
[KAFKA-757] - System Test Hard Failure cases : "Fatal error during KafkaServerStable startup" when hard-failed broker is re-started
[KAFKA-758] - startHighWaterMarksCheckPointThread is never called
[KAFKA-767] - Message Size check should be done after assigning the offsets
[KAFKA-768] - broker should exit if hitting exceptions durin startup
[KAFKA-769] - On startup, a brokers highwatermark for every topic partition gets reset to zero
[KAFKA-770] - KafkaConfig properties should be verified in the constructor
[KAFKA-772] - System Test Transient Failure on testcase_0122
[KAFKA-776] - Changing ZK format breaks some tools
[KAFKA-779] - Standardize Zk data structures for Re-assign partitions and Preferred replication election
[KAFKA-785] - Resolve bugs in PreferredReplicaLeaderElection admin tool
[KAFKA-786] - Use "withRequiredArg" while parsing jopt options in all tools
[KAFKA-793] - Include controllerId in all requests sent by controller
[KAFKA-798] - Use biased histograms instead of uniform histograms in KafkaMetricsGroup
[KAFKA-800] - inSyncReplica in Partition needs some tweaks
[KAFKA-801] - Fix MessagesInPerSec mbean to count uncompressed message rate
[KAFKA-804] - Incorrect index in the log of a follower
[KAFKA-807] - LineMessageReader doesn't correctly parse the key separator
[KAFKA-809] - Dependency on zkclient 0.1 (redundant) prevents building in IntelliJ
[KAFKA-811] - Fix clientId in migration tool
[KAFKA-813] - Minor cleanup in Controller
[KAFKA-825] - KafkaController.isActive() needs to be synchronized
[KAFKA-826] - Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[KAFKA-827] - improve list topic output format
[KAFKA-828] - Preferred Replica Election does not delete the admin path on controller failover
[KAFKA-830] - partition replica assignment map in the controller should be a Set
[KAFKA-832] - 0.8 Consumer prevents rebalance if consumer thread is blocked or slow
[KAFKA-840] - Controller tries to perform preferred replica election on failover before state machines have started up
[KAFKA-846] - AbstractFetcherThread should do shallow instead of deep iteration
[KAFKA-856] - Correlation id for OffsetFetch request (#2) always responds with 0
[KAFKA-861] - IndexOutOfBoundsException while fetching data from leader
[KAFKA-866] - Recover segment does shallow iteration to fix index causing inconsistencies
[KAFKA-871] - Rename ZkConfig properties
[KAFKA-872] - Socket server does not set send/recv buffer sizes
[KAFKA-880] - NoLeaderPartitionSet should be cleared before leader finder thread is started up
[KAFKA-884] - Get java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J when stopping kafka brokers
[KAFKA-895] - Protocol documentation is not clear about requiredAcks = 0.
[KAFKA-899] - LeaderNotAvailableException the first time a new message for a partition is processed.
[KAFKA-900] - ClosedByInterruptException when high-level consumer shutdown normally
[KAFKA-903] - [0.8.0 - windows] FATAL - [highwatermark-checkpoint-thread1] (Logging.scala:109) - Attempt to swap the new high watermark file with the old one failed
[KAFKA-905] - Logs can have same offsets causing recovery failure
[KAFKA-907] - controller needs to close socket channel to brokers on exception
[KAFKA-914] - Deadlock between initial rebalance and watcher-triggered rebalances
[KAFKA-916] - Deadlock between fetcher shutdown and handling partitions with error
[KAFKA-919] - Disabling of auto commit is ignored during consumer group rebalancing
[KAFKA-920] - zkclient jar 0.2.0 is not compatible with 0.1.0
[KAFKA-921] - Expose max lag mbean for consumers and replica fetchers
[KAFKA-927] - Integrate controlled shutdown into kafka shutdown hook
[KAFKA-937] - ConsumerFetcherThread can deadlock
[KAFKA-938] - High CPU usage when more or less idle
[KAFKA-940] - Scala match error in javaapi.Implicits
Improvement
[KAFKA-77] - Implement "group commit" for kafka logs
[KAFKA-133] - Publish kafka jar to a public maven repository
[KAFKA-134] - Upgrade Kafka to sbt 0.11.3
[KAFKA-139] - cross-compile multiple Scala versions and upgrade to SBT 0.12.1
[KAFKA-155] - Support graceful Decommissioning of Broker
[KAFKA-165] - Add helper script for zkCli.sh
[KAFKA-181] - Log errors for unrecognized config options
[KAFKA-187] - Add Snappy Compression as a Codec and refactor CompressionUtil and option on startup to select what the default codec
[KAFKA-193] - use by name parameter helper for logging and trait to include lazy logging and refactor code to use the new LogHelper
[KAFKA-246] - log configuration values used
[KAFKA-253] - Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster
[KAFKA-258] - Remove broker.id from the broker.list config in the Producer
[KAFKA-267] - Enhance ProducerPerformance to generate unique random Long value for payload
[KAFKA-271] - Modify new FetchResponse object to remove the initial offset field
[KAFKA-281] - support multiple root log directories
[KAFKA-285] - Increase maximum value of log.retention.size
[KAFKA-296] - Update Go Client to new version of Go
[KAFKA-311] - Allow KafkaLog4jAppender to take in a configurable producer.type
[KAFKA-314] - Go Client Multi-produce
[KAFKA-323] - Add the ability to use the async producer in the Log4j appender
[KAFKA-324] - enforce broker.id to be a non-negative integer
[KAFKA-325] - revisit broker config in 0.8
[KAFKA-349] - Create individual "Response" types for each kind of request and wrap them with "BoundedByteBufferSend", remove "xxResponseSend" types for all requests except "FetchRequest"
[KAFKA-365] - change copyright in NOTICE to 2012
[KAFKA-366] - add jmx beans in broker to track # bytes in consumer
[KAFKA-368] - use the pig core jar from maven instead of distributing it
[KAFKA-393] - Add constructor for message which takes both byte array offset and length
[KAFKA-408] - ProducerPerformance does not work with all producer config options
[KAFKA-437] - Unused var statement in ZookeeperConsumerConnectorTest
[KAFKA-439] - @returns was used in scala doc when it should have been @return
[KAFKA-505] - Remove errorcode from TopicMetaDataResponse
[KAFKA-548] - remove partition from ProducerRequestPartitionData and FetchResponsePartitionData
[KAFKA-581] - provides windows batch script for starting Kafka/Zookeeper
[KAFKA-632] - ProducerRequest should take ByteBufferMessageSet instead of MessageSet
[KAFKA-638] - remove ProducerShell
[KAFKA-667] - Rename .highwatermark file
[KAFKA-675] - Only bind to the interface declared in the 'hostname' config property
[KAFKA-699] - Disallow clients to set replicaId in FetchRequest
[KAFKA-733] - Fat jar option for build, or override for ivy cache location
[KAFKA-762] - Improve second replica assignment
[KAFKA-763] - Add an option to replica from the largest offset during unclean leader election
[KAFKA-812] - Support deep iteration in DumpLogSegments tool
[KAFKA-850] - add an option to show under replicated partitions in list topic command
[KAFKA-931] - make zookeeper.connect a required property
New Feature
[KAFKA-50] - kafka intra-cluster replication support
[KAFKA-188] - Support multiple data directories
[KAFKA-202] - Make the request processing in kafka asynchonous
[KAFKA-203] - Improve Kafka internal metrics
[KAFKA-235] - Add a 'log.file.age' configuration parameter to force rotation of log files after they've reached a certain age
[KAFKA-429] - Expose JMX operation to set logger level dynamically
[KAFKA-475] - Time based log segment rollout
[KAFKA-545] - Add a Performance Suite for the Log subsystem
[KAFKA-546] - Fix commit() in zk consumer for compressed messages
Task
[KAFKA-93] - Change code header to follow standard ASF source header
[KAFKA-317] - Add support for new wire protocol to Go client
[KAFKA-341] - Create a new single host system test to validate all replicas on 0.8 branch
[KAFKA-348] - rebase 0.8 branch from trunk
[KAFKA-380] - Enhance single_host_multi_brokers test with failure to trigger leader re-election in replication
[KAFKA-440] - Create a regression test framework for distributed environment testing
[KAFKA-526] - System Test should remove the top level data log directory
[KAFKA-594] - Update System Test due to new argument "--sync" in ProducerPerformance
[KAFKA-605] - System Test - Log Retention Cases should wait longer before getting the common starting offset in replica log segments
[KAFKA-688] - System Test - Update all testcase_xxxx_properties.json for properties keys uniform naming convention
[KAFKA-737] - System Test : Disable shallow.iterator in Mirror Maker test cases to make compression work correctly
[KAFKA-791] - Fix validation bugs in System Test
[KAFKA-792] - Update multiple attributes in testcase_xxxx_properties.json
[KAFKA-819] - System Test : Add validation of log segment index to offset

kafka-0.7.0-incubating-candidate-9

kafka-0.7.1-incubating-candidate-1

kafka-0.7.1-incubating-candidate-2

kafka-0.7.1-incubating-candidate-3

kafka-0.7.2-incubating-candidate-1