Autoscaler on OpenShift Dedicated


At the time of writing, OpenShift dedicated (v3.5.5.31) only supports autoscaling based on the CPU metrics. So the first step is to ensure the CPU request is in place. It can be setup in the section of container spec. e.g

spec:
  containers:
      image: 'poc/web'    
      name: my-web
      ports:
        - containerPort: 3000
      resources:
        requests:
          cpu: 200m
        limits:
          memory: 200Mi

The second step is to setup an autoscaler policy and associate it with the deployment configuration. The following example is to setup an autoscaler for adjusting the replicas which sits behind the my-web service. New pods will spin up if the existing pods CPU usage is higher than 80%. The maximum pods number is 5.

apiVersion: extensions/v1beta1
kind: HorizontalPodAutoscaler
metadata:
  name: my-web
spec:
  scaleRef:
    kind: DeploymentConfig 
    name: my-web
    apiVersion: v1 
    subresource: scale
  minReplicas: 1 
  maxReplicas: 5 
  cpuUtilization:
    targetPercentage: 80   

Unfortunately, our OpenShift dedicated cluster is not 100% functional at this moment, as the autoscaler is unable to get the current CPU usage from heapster due to some cluster configurations I believe – unable to get metrics for resource cpu: failed to get heapster service: an error on the server (“x509: certificate signed by unknown authority”) has prevented the request from succeeding (get services https:heapster:)

The metrics collection system is part of the cluster infrastructure. As a managed service vendor, RedHat should ensure all the critical components are monitored and tested. Personally, I feel very disappointed at their support quality since we start to use OpenShift dedicated. And it seems they only have first level support in APAC region, so for most of the time we have to wait their North America operation team to respond.  For this particular one,  a ticket has been logged, I am still waiting for their reply. I found something that may be related, but need RedHat to confirm it.

https://access.redhat.com/solutions/3137101
https://access.redhat.com/solutions/2994711

oc describe hpa/my-web
Name:				my-web
Namespace:			poc
Labels:				
Annotations:			
CreationTimestamp:		Fri, 20 Oct 2017 15:12:42 +1100
Reference:			DeploymentConfig/my-web
Target CPU utilization:		80%
Current CPU utilization:	
Min replicas:			1
Max replicas:			5
Events:
  FirstSeen	LastSeen	Count	From				SubObjectPath	Type		Reason			Message
  ---------	--------	-----	----				-------------	--------	------			-------
  1h		57m		8	{horizontal-pod-autoscaler }			Normal		MetricsNotAvailableYet	unable to get metrics for resource cpu: failed to get heapster service: an error on the server ("x509: certificate signed by unknown authority") has prevented the request from succeeding (get services https:heapster:)
  57m		16s		118	{horizontal-pod-autoscaler }			Warning		FailedGetMetrics	unable to get metrics for resource cpu: failed to get heapster service: an error on the server ("x509: certificate signed by unknown authority") has prevented the request from succeeding (get services https:heapster:)
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s