FairScheduler with preemption -- How to interpret Steady Fair Share vCore

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

FairScheduler with preemption -- How to interpret Steady Fair Share vCore

Aaron Niskode-Dossett
Hello,

I'm running CDH 5.10.0 and have Fair Scheduler preemption enabled.  When I look at the different YARN queues in the YARN UI I see information like:

Min Resources: <memory:0, vCores:0>
Max Resources: <memory:73850060, vCores:9920>
Steady Fair Share: <memory:1918183, vCores:0>
Instantaneous Fair Share: <memory:0, vCores:0>
Preemptable: true

----or------

Used Resources: <memory:31343616, vCores:8745>
Num Active Applications: 8
Num Pending Applications: 0
Min Resources: <memory:0, vCores:0>
Max Resources: <memory:73850060, vCores:9920>
Steady Fair Share: <memory:10789782, vCores:0>
Instantaneous Fair Share: <memory:73850064, vCores:0>


The "Steady Fair Share" has always shown "vCores:0".   For various reasons, my cluster is more likely to be vCore constrained that memory constrained.  If the Steady Fair Share for vCore is 0, and there are no free vCores does that mean that preemption would not free up vCores for an application that needed that to run?  

My YARN queues are configured with options like:

<queue name="production">
            <weight>.45</weight>
            <fairSharePreemptionThreshold>0.9</fairSharePreemptionThreshold>
            <fairSharePreemptionTimeout>60</fairSharePreemptionTimeout>
...
</queue>

Do I need to anything else to enable vCore preemption?

Thanks! -Aaron