HADOOP-18140 (SPARK-17969). "Tar of compressed files (zipped, gzipped) fails with 'File exists' exception", has been fixed in Apache Spark 1.0.1. Users can upgrade to Spark 1.0.1.
Upside of Upgrade to Current Stable Apache Hadoop Versions Apache Hadoop does not have any default user accounts that have elevated privileges. A remote attacker can only exploit these vulnerabilities if they have already obtained access to your network. By upgrading to the current stable Apache Hadoop versions, you can ensure that you are running the most secure software.
Apache Hadoop YARN Versions
Apache Hadoop YARN Versions
According to the Apache Hadoop YARN website, the latest stable release of Apache Hadoop is version 2.7.0.
Apache Hadoop Versioning
Apache Hadoop is released in three different versions: Foundation, which is the original release of Apache Hadoop; Stable, which is the current stable version of Apache Hadoop; and Incubating, which is the next version of Apache Hadoop that's still in active development.
The current stable versions are available as both binary releases and source code releases. The binaries are not supported by the community, but they're easier to install and use than the source code releases. For example, when upgrading from Foundation to Stable, there's a transitional period in which you must run an older release alongside a newer one. When upgrading from Stable to Incubating, you must compile your own binaries from source code.
Apache Spark and Hadoop Versions compatibility
The current Apache Spark release is 1.0.1 and the current stable Hadoop release is 2.6.1. Although the two versions are compatible, all users are encouraged to upgrade to the most current version of both Apache Spark and Hadoop.
Apache Hadoop Versions and their Limitations
The newest version of Apache Hadoop, Spark 1.0.1, fixes this vulnerability.
The old versions of Apache Hadoop have known vulnerabilities that could allow for a remote attacker to exploit these vulnerabilities and gain access to your network.
As a general rule for all software, it's best to upgrade to the current stable version.
Apache Hadoop YARN Service Node Software Upgrade
The key improvement is to provide fine-grained privilege management for YARN service node users. For example, users with the role of "resource manager" will be able to only read data from a particular queue or access a specific directory and not have access to other queues or directories. Another improvement concerns the Hadoop Security Project (HSP) which will provide enhanced security in remote log hosts by default.
Timeline
Published on: 08/04/2022 15:15:00 UTC
Last modified on: 08/10/2022 15:36:00 UTC