Large Internet services companies like Google, Yahoo, and Facebook use the MapReduce programming model to process log data. MapReduce is designed to work on data stored in a distributed filesystem like Hadoop's HDFS. As a result, a number of companies have developed log collection systems that write to HDFS. These systems have a number of common weaknesses, induced by the semantics of the filesystem. They impose a delay, often several minutes, before data is available for processing. They are difficult to integrate with existing applications. They cannot reliably handle concurrent failures. We present a system, called Chukwa, that adds the needed semantics for log collection and analysis. Chukwa uses an end-to-end delivery model that leverages local on-disk log files when possible, easing integration with legacy systems. Chukwa offers a choice of delivery models, making subsets of the collected data available promptly for clients that require it, while reliably storing a copy in HDFS. We demonstrate that our system works correctly on a 200-node testbed and can collect in excess of 200 MB/sec of log data. We supplement these measurements with a set of case studies.