[prev in list] [next in list] [prev in thread] [next in thread] 

List:       mesos-commits
Subject:    [1/2] mesos git commit: Updated `Slave::getAvailable()` to return all shared resources.
From:       bmahler () apache ! org
Date:       2018-07-25 23:20:01
Message-ID: 6064af06850048fa941e77426d64b36c () git ! apache ! org
[Download RAW message or body]

Repository: mesos
Updated Branches:
  refs/heads/master 56d421d4d -> eeb74c036


Updated `Slave::getAvailable()` to return all shared resources.

Currently, `HierarchicalAllocatorProcess::Slave::getAvailable()`
only exposes the unallocated portion of shared resources. However,
we currently always consider a copy of shared resources to be
available for allocation.

This helps us acheive some cleanup of one-off shared resources logic
in the allocation loop.

Review: https://reviews.apache.org/r/67826/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/a9d50efd
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/a9d50efd
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/a9d50efd

Branch: refs/heads/master
Commit: a9d50efd63bf4cc47a49ffe3feaef87dfc0ffa7a
Parents: 56d421d
Author: Meng Zhu <mzhu@mesosphere.io>
Authored: Wed Jul 25 15:57:47 2018 -0700
Committer: Benjamin Mahler <bmahler@apache.org>
Committed: Wed Jul 25 16:15:30 2018 -0700

----------------------------------------------------------------------
 src/master/allocator/mesos/hierarchical.hpp | 41 +++++++++++++++++-------
 1 file changed, 30 insertions(+), 11 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/a9d50efd/src/master/allocator/mesos/hierarchical.hpp
----------------------------------------------------------------------
diff --git a/src/master/allocator/mesos/hierarchical.hpp b/src/master/allocator/mesos/hierarchical.hpp
index c1a6789..02c63b1 100644
--- a/src/master/allocator/mesos/hierarchical.hpp
+++ b/src/master/allocator/mesos/hierarchical.hpp
@@ -372,14 +372,10 @@ protected:
         capabilities(_capabilities),
         activated(_activated),
         total(_total),
-        allocated(_allocated)
+        allocated(_allocated),
+        shared(_total.shared())
     {
-      // In order to subtract from the total,
-      // we strip the allocation information.
-      Resources allocated_ = allocated;
-      allocated_.unallocate();
-
-      available = total - allocated_;
+      updateAvailable();
     }
 
     const Resources& getTotal() const { return total; }
@@ -390,6 +386,7 @@ protected:
 
     void updateTotal(const Resources& newTotal) {
       total = newTotal;
+      shared = total.shared();
 
       updateAvailable();
     }
@@ -461,7 +458,21 @@ protected:
       Resources allocated_ = allocated;
       allocated_.unallocate();
 
-      available = total - allocated_;
+      // Calling `nonShared()` currently copies the underlying resources
+      // and is therefore rather expensive. We avoid it in the common
+      // case that there are no shared resources.
+      //
+      // TODO(mzhu): Ideally there would be a single logical path here.
+      // One solution is to have `Resources` be copy-on-write such that
+      // `nonShared()` performs no copying and instead points to a
+      // subset of the original `Resource` objects.
+      if (shared.empty()) {
+        available = total - allocated_;
+      } else {
+        // Since shared resources are offerable even when they are in use, we
+        // always include them as part of available resources.
+        available = (total.nonShared() - allocated_.nonShared()) + shared;
+      }
     }
 
     // Total amount of regular *and* oversubscribed resources.
@@ -480,14 +491,22 @@ protected:
     // hasn't reregistered. See MESOS-2919 for details.
     Resources allocated;
 
-    // We track the total and allocated resources on the slave, the
-    // available resources are computed as follows:
+    // We track the total and allocated resources on the slave to
+    // avoid calculating it in place every time.
     //
-    //   available = total - allocated
+    // Note that `available` always contains all the shared resources on the
+    // agent regardless whether they have ever been allocated or not.
+    // NOTE, however, we currently only offer a shared resource only if it has
+    // not been offered in an allocation cycle to a framework. We do this mainly
+    // to preserve the normal offer behavior. This may change in the future
+    // depending on use cases.
     //
     // Note that it's possible for the slave to be over-allocated!
     // In this case, allocated > total.
     Resources available;
+
+    // We keep a copy of the shared resources to avoid unnecessary copying.
+    Resources shared;
   };
 
   hashmap<SlaveID, Slave> slaves;

[prev in list] [next in list] [prev in thread] [next in thread]