We are excited to announce the release of Tachyon v0.8. This is the largest release to date with numerous new features, integrations, and stability improvements. Highlights of major features and updates are listed below.
Downloads for the release can be found here: Downloads.
Unified Namespace
Tachyon v0.8 provides an interface for mounting multiple data sources into the Tachyon namespace, taking advantage of Tachyon’s ability to interoperate with different underlying storage systems such as HDFS, S3, GlusterFS, or Swift. Information about mounted data sources is managed centrally by the Tachyon master service, facilitating reconfiguration.
Improved File System Client API
Tachyon v0.8 significantly improves the native file system API, making it more similar to Java 7’s Files API. The updated API exposes a minimal set of methods to interact with Tachyon as TachyonFileSystemCore and builds more complex operations on top of these in TachyonFileSystem.
Transparent Naming
Tachyon v0.8 introduces transparent naming, a feature that preserves Tachyon paths in the underlying storage system namespace. In particular, with transparent naming, paths and directory hierarchies created in Tachyon are reflected in the underlying storage system and vice versa.
Mesos Integration
Tachyon v0.8 provides integration with the Mesos resource manager. In particular, the release provides an implementation of a Tachyon Mesos framework that can be used to start an instance of Tachyon on any Mesos-managed cluster, greatly simplifying Tachyon deployment.
Yarn Integration
Tachyon v0.8 also has an option to start Tachyon with Hadoop YARN. This integration provides a YARN client to apply for resources (e.g., Memory and CPU) from YARN resource manager to launch a Tachyon instance as a YARN application. This will simplify the deployment and resource management when running Tachyon in an environment of YARN cluster.
Configuration Improvements
Tachyon configuration system has been greatly improved in v0.8 to be more user-friendly. Particularly, passing down Tachyon-specific configuration parameters to Tachyon client when running third-party computation frameworks like Hadoop MapReduce becomes much easier.
New Documentation Layout
Tachyon v0.8 comes with a much easier to navigate user interface for the documentation. The pages are divided into sections based on component integrations such as under storage and compute framework. The documentation also provides more detailed tutorials on running Tachyon as part of different stacks.
Space Reserver
Tachyon v0.8 provides the option of enabling a space reserver on each worker. The space reserver periodically monitors the space usage of each storage tier on the worker and starts off background data eviction and migration when the storage usage exceeds a certain user defined threshold. Enabling the space reserver benefits workloads with bursty writes and when synchronous data eviction and migration would be detrimental to performance.
LRFU Eviction Policy
Tachyon v0.8 comes prepackaged with Least Recently/Frequently Used eviction policy implementation. Based on the user configuration, the policy will behave between a spectrum of Least Recently Used and Least Frequently Used. Users can simply modify the configuration parameters to achieve optimal performance for their workloads without the need to write custom code.
File TTL
Tachyon v0.8 allows users to specify a Time To Live for new files they create. After the TTL expires, the file will automatically be deleted in Tachyon and under storage. The TTL concept provides an easier way to manage temporary files without having to maintain additional processes.
Deploy Module Improvement
The deploy module has been updated to allow for deploying Tachyon with a variety of frameworks such as Yarn and Mesos as well as fault tolerant mode. Tachyon can also be seamlessly deployed with S3 as the under storage. Finally, when deploying on AWS, spot instances can now be leveraged.
Acknowledgements
We would like to thank the following members of the community for their contributions to Tachyon 0.8. The release would not have been possible without your efforts!
- Akshay Pundle – Code improvements in Tachyon master
- Amelia Wong – Documentation improvements in Tachyon client
- Andrea Reale – Code improvements and bug fixes in Tachyon client and Tachyon master
- Andrew Audibert – Code improvements in Tachyon master
- Aslan Bekirov – Code improvements in Tachyon – Mesos integration
- Bin Fan – Fine grained locking in Tachyon worker, Tachyon – Yarn integration
- Calvin Jia – Code improvements in Tachyon client and master
- Carson Wang – Metrics collection, Code improvements in Tachyon client and worker
- Chang Cheng – Code improvements in Tachyon master, Deploy module improvements
- Charlie Chen – Code improvements in Tachyon thrift
- Chen Jin – Code improvements in Tachyon client
- Chen Tian – Code improvements in Tachyon master and client, Deploy module improvements
- Chris Lu – Code improvements in Tachyon client
- Daniel Yu – Code improvements in Tachyon client and master
- David Zhu – Code improvements in Tachyon client, Deploy module improvements
- Dong Chen – Basic user authentication
- Dong Lin – Code improvements in Tachyon worker
- Fabio Iwakoshi – Code improvements in Tachyon web server
- Gautam Kumar – Code improvements in Tachyon master and worker
- Gene Pang – Modularization of Tachyon master, Configuration updates
- Guoquan Shen – Basic user authentication
- Haochen Zhang – Code improvement in Tachyon – S3 integration
- Haoyuan Li – Documentation and code structure improvements
- Haozhi Sun – Code improvements in Tachyon client
- Jie Zhou – Code improvements in Tachyon common
- Jiri Simsa – Mount and transparent naming, Tachyon – Mesos integration
- Kai Ren – Code improvements in Tachyon client
- Kelvin Chu – Code improvements in Tachyon client
- Ku Liu – Code improvements in Tachyon tests
- Leo Wang – Code improvements in Tachyon worker
- Liangliang Gu – Checkstyle improvements
- Luo Li – Code improvements in Tachyon client and shell
- Ma Biao – Code improvements in Tachyon client
- Manu Goyal – Exception handling improvements, code improvements in Tachyon client
- Mauro Cortellazzi – Code improvements in Tachyon client
- Mingfei Shi – Space Reserver
- Nan Dun – Code improvements in Tachyon client
- Ning Qu – Code improvements in Tachyon workers
- Ozturk Gokal – Code improvements in Tachyon servers
- Peng Shu – LRFU Eviction policy
- Pengfei Xuan – Improved network configuration support
- Pradeep Bhadani – Code improvements in Tachyon worker
- Pranay Tonpay – Code improvements in Tachyon examples
- Qianhao Dong – Code improvements in Tachyon master
- Qifan Pu – BlockMetadata Manager views in Tachyon worker
- Qinghao Dai – Code improvements in Tachyon worker
- Rean Griffith – Code improvements in Tachyon worker and tests
- Roberto Bentivoglio – Code and documentation improvements in Tachyon master and tests
- Ryan Huang – Code and documentation improvements in Tachyon worker
- Saverio Veltri – Code improvements in Tachyon tests
- Shiqing Fan – Code improvements in Tachyon master
- Sidharth Goel – Documentation improvements in Tachyon master
- Sina Sheikholeslami – Code improvements in Tachyon client
- Steven Chen – Code improvements in Tachyon examples
- Sun Dapeng – Basic user authentication
- Tianyin Xu – Wild card input for Tachyon shell
- Timothy O’Dell – Code improvements to Tachyon master
- Vu Pham – Code improvements in Tachyon client
- Wang Chao – Code improvement in Tachyon master
- Weide Zhang – File TTL
- Xiaoyi Wang – Code improvement in Tachyon worker
- Xin Wang – Code improvement in Tachyon master
- Yang Fan – Code improvements in Tachyon client and master
- Yu Zhang – Code improvements in Tachyon client
- Yupeng Fu – Code improvements in Tachyon master
- Zhen Li – Code improvements in Tachyon worker
- Zhongyi Xie – Code improvements in Tachyon client and master
- Zongwei Zhou – Code improvements in Tachyon common