Datacenter resource management / schedulers - Implementations
-
YARN - Apache Hadoop YARN (Yet Another Resource Negotiator) is a cluster management technology
YARN - github
Apache Hadoop Yarn - Homesite
Apache Hadoop Yarn - Hortonworks
YARN is now characterized as a large-scale, distributed operating system for big data applications.
YARN - DOC
2012 - Hortonworks - Introducing Apache Hadoop YARN
2013 - Hortonworks - Philosophy behind YARN resource management
2012 - Hortonworks - Apache Hadoop YARN – Background and an Overview
2012 - Hortonworks - Apache Hadoop YARN – Concepts and applications
2012 - Hortonworks - Apache Hadoop YARN – Resource Manager
2012 - Hortonworks - Apache Hadoop YARN – Node Manager -
Kubernetes (License: Apache-2.0)
Kubernetes - github
Kubernetes - Documentation
Kubernetes is an open source system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications. Kubernetes is hosted by the Cloud Native Computing Foundation (CNCF). Kubernetes builds upon a decade and a half of experience at Google running production workloads at scale using a system called Borg, combined with best-of-breed ideas and practices from the community. -
Minikube
Running Kubernetes Locally via Minikube
Minikube is a tool that makes it easy to run Kubernetes locally. Minikube runs a single-node Kubernetes cluster inside a VM on your laptop for users looking to try out Kubernetes or develop with it day-to-day. -
MESOS - distributed systems kernel
Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elastic Search) with API’s for resource management and scheduling across entire datacenter and cloud environments. -
MiniMesos - Testing infrastructure for Mesos frameworks
The experimentation and testing tool for Apache Mesos -
Bistro (License: BSD) - A fast, flexible toolkit for scheduling and running distributed tasks
Bistro - github
A toolkit for making services that schedule and execute tasks. Bistro is an engineer's tool — your clients need to do large amounts of computation, and your goal is to make a system that handles them easily, performantly, and reliably.
Bistro - DOC
Bistro - Getting started
Bistro: Scheduling Data-Parallel Jobs Against Live Production Systems -
Firmanent (LICENSE: Apache 2.0)
Firmament is based on a graph-theoretic construct called a flow network, over which it runs a minimum-cost optimization. Unlike heuristic-driven schedulers, Firmament's approach is guaranteed to find the policy-optimal assignment of work to cluster resources. Flexible policy, scalability to large clusters, and high-quality scheduling decisions need not be exclusive: Firmament's principled approach has you covered. Firmament works stand-alone, or within your cluster manager. - OpenSVC