2

Container Engine for Kubernetes and

OCI Registry Service

Copyright © 2018, Oracle and/or its affiliates. All rights reserved.

Objectives
After completing this lesson, you should be able to:
• Describe our Container based Infrastructure Strategy
• Describe the OCI Container Engine for Kubernetes
• Describe the OCI Registry Service
• Launch a Kubernetes Cluster on OCI and understand basic use of OCI
registry service

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2-2
Agenda

1 Container Based Infrastructure Strategy

2 OCI Container Engine for Kubernetes

3 OCI Registry Service

4
OKE/OCIR Competitive and Roadmap Questions

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

History and Multi-Dimensional Evolution of Computing

Copyright
Copyright © 2017 Oracle © 2018,
and/or its affiliates. All Oracle and/or
rights reserved its affiliates. All rights reserved. 2-4
Oracle Strategy for Container Based Infrastructure
Deliver a container based capabilities that are complete, integrated and open
• Orchestration/Scheduling, CI/CD, Management/Operations, Analytics/Introspection
• With application development platform for serverless and microservices
Based on community driven open source technology
• Investing in open source communities and foundations (Kubernetes, Docker, CNCF) via
engineering resources, code contributions & sponsorship
Differentiated on quality of service and operational excellence
• Full, transparent management
• Deployed to Oracle Cloud Infrastructure
• Enterprise grade security, HA and governance

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2-5
Oracle & Open Source and Community for Containers
Oracle’s participation in open source community
• Active Participation – Cloud Native Compute
Unforked
Foundation and Kubernetes fn Open Source
• No forked code – straight from the source
• Continue precedence of Java, MySQL, Linux
Lead by example Active
• Oracle software on Docker Store Community
Participation
• Kubernetes engineering in CNCF
Innovate in open source
• Utilities like K8S installer, smith, railcar, crashcart Innovation in
Sponsor & contribute to key conferences Open Source
smith crashcart railcar
• DockerCon, Kubecon, CoreOS Fest, others

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2-6
Container Native Application Development Capabilities
Build, Deploy, Operate Container Based Applications

OCI Container Engine for Kubernetes (OKE) Fully Managed Standard Kubernetes Service

Oracle Cloud Infrastructure Registry (OCIR) Docker Compliant Container Image Registry

Oracle Container Pipelines

Oracle Developer Cloud Continuous Integration and Delivery Pipeline

Fn Project
Open Source Serverless Functions Framework

Open Source Extensibility OKE supports HELM for easy deployments

https://github.com/kubernetes/charts/tree/master/stable

Copyright
Copyright © 2018, Oracle and/or its©affiliates.
2018, All Oracle and/or its
rights reserved. affiliates. All rights reserved. 2-7
Oracle Cloud Infrastructure and Kubernetes
Roll Your Own, Pre-Built Installer, Managed Service

Quickstart Experience OCI Container Engine

OCI OSS Terraform Installer on GitHub for Kubernetes

DIY Container Self Managed Enterprise Class Managed

Management Kubernetes Service Kubernetes Service

IaaS CaaS

Copyright © 2017 Oracle and/or its affiliates.

Copyright © 2018, All Oracle
rights reserved
and/or its affiliates. All rights reserved. 2-8
DIY - Terraform Kubernetes Installer for OCI
Open Source OCI Kubernetes installer, based on
Terraform
• Oracle developed for Kubernetes on OCI
• Available now on Github -
https://github.com/oracle/terraform-kubernetes-instal
ler

Key Highlights
• Highly available Kubernetes cluster configured in
your OCI tenancy and compartment
• Creates VCN, subnets, LBs and instances for
control plane
• Specify number and shape of nodes for your cluster
• Scale your cluster as needed
Available on Oracle Github!
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 9
Kubernetes Challenges

• Managing, maintaining, upgrading Kubernetes Control

Plane
– API Server, etcd, scheduler etc….
• Managing, maintaining, upgrading Kubernetes Data Plane
– In place upgrades, deploy parallel cluster etc….
• Figuring out container networking & storage
– Overlays, persistent storage etc… - it should just
work
• Managing Teams
– How do I manage & control team access to my
clusters?
• CI/CD Integration
– How do I drive automated testing and conditional
release into my application lifecycle?
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 10
Oracle Cloud Infrastructure – Managed Container Engine and
Container Registry
OCI - Registry OCI – Kubernetes Customer’s OCI
Engine Account/Tenancy

Cluster Management
VM based Clusters and Nodes
HA - Masters/etcd
Encryption for Data in across multiple ADs
Transit (SSL) and at Rest Bare Metal Clusters and Nodes
Container Engine
Dashboard

Oracle Cloud Infrastructure

Oracle Managed Customer Managed

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Restricted 11
 Detail: OCI Container Engine for Kubernetes and Registry
An Open, Fully-Managed Kubernetes Platform & Private Registry

Build
CI/CD
system Test
of choice
- ie Test

Jenkins, Test
OCI Registry OCI Container Engine for
Wercker,
Kubernetes
etc. Push

VCN
AD 1 AD 2 AD 3

LB
Exposed
Kubernetes Service PV K8S Cluster

VM
Node Pool

Pods Node Pool

BM

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 12

OKE Engine Differentiators
Cloud & Container Native
• End-to-end container Lifecycle
Management
 Build, test, deploy, operate
• Registry Integration
 Full Docker v2 compatible registry
for private images
• Standard Kubernetes
 Deploy standard & open upstream
Kubernetes versions for
compatibility
• OCI Aware Integrated Plugins
 Persistent Storage
 Load Balancing
Developer Friendly Enterprise Ready
• One Click Clusters • Full Bare Metal Performance
 Combine Kubernetes with VM and
 Simplify cluster setup
BM shapes for raw performance
• Full REST API and CLI
• HA Managed Control Plane
 Create and scale clusters through
full REST API • Multi-AD/Zone Support
• Built In Cluster Add-Ons  Deploy resilient Kubernetes apps
 Kubernetes Dashboard, DNS & • Node Pools & Self-Healing
Helm Clusters
• Open Standards  Create and scale clusters of
different shapes; maintain cluster
 Docker Based Runtime size in face of node failures
 Worker Node SSH Access
• Team Based Access Controls
 Control team access and
permissions to clusters
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 13
 OCI Registry - OCIR

• High availability Docker v2 container registry service on

Oracle Cloud Infrastructure
• Full integration with OKE Push Deploy
• Stores Docker Images in Private Repositories Registry

• Automatic Org Image Layer De-duplication Any CI/CD

• Co-located regionally with Container Engine for low

latency Docker image deploys

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 14

OCI Container Engine for Kubernetes

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 15

Pre-requisites for OKE (1) - Service Limits for tenancy

• Must have compute Instance Quota (Required) – to launch k8s worker nodes
in an AD or across ADs for HA
• Block Volume Quota – Only required if you want to create k8s persistent
volumes
• Load Balancer Quota – Only required if you want to distribute traffic between
worker nodes

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 16

Pre-requisites for OKE (2) – Required IAM policies

• Required Policy in the root compartment of your tenancy

• allow service OKE to manage all-resources in tenancy
• To launch a K8s cluster, user must be either part of the Admin group or a
group to which a policy grants the appropriate Container Engine for
Kubernetes permissions.
• Policies can be created for users which are not part of the admin group
• For Example: To enable users in group ’dev-team’ to perform any operation on
cluster-related resources  allow group dev-team to manage
cluster-family in tenancy
• Note: if users will be using the Console to create and update clusters,
polices must also grant the dev-team group the Networking permissions
VCN_READ and SUBNET_READ.
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 17
Pre-requisites for OKE (3) – Basic Virtual Cloud Network Config
• An Existing VCN with following
• Internet Gateway
• Route table with default route to IGW
• K8s worker node subnets – atleast three subnets in different ADs for HA
• LBs Subnets – 2 subnets in different ADs for OCI Public LB
• Separate Security Lists for K8s Worker Nodes Subnets and LB Subnets
• Security Lists for K8s worker Nodes Subnets
– Stateless ingress and egress rules that allow all traffic between the different worker node
subnets
– stateless ingress and egress rules that allow all traffic between worker node subnets and
load balancer subnets
– ingress rules to allow Container Engine for Kubernetes to access worker nodes on port 22
from 130.35.0.0/16 and 138.1.0.0/17
– an egress rule that allows all outbound traffic to the internet

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 18

Kubernetes Cluster Creation

• Name – Name of the K8s Cluster

• Version - The version of Kubernetes to run on the master node of the cluster
• VCN - The name of an existing virtual cloud network that has been configured for cluster
creation and deployment
• Kubernetes Service LB Subnets: The two subnets configured to host load balancers.
• Kubernetes Service CIDR Block (Optional): The available group of network addresses
that can be exposed as Kubernetes services (ClusterIPs), expressed as a single,
contiguous IPv4 CIDR block. For example, 10.96.0.0/16. Must not overlap with VCN
CIDR
• Pods CIDR Block (Optional): The available group of network addresses that can be
allocated to pods running in the cluster, expressed as a single, contiguous IPv4 CIDR
block. For example, 10.244.0.0/16. Must not overlap with VCN CIDR
• Kubernetes Dashboard and Helm are enabled by default

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 19

Kubernetes Cluster Creation

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 20

Kubernetes Cluster Details and Node Pools

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 21

Kubernetes Worker Nodes - Nodepools
• Name – Name of the node pool
• Version - The version of Kubernetes to run on each worker node in the node pool. By
default, the version of Kubernetes specified for the master node is selected. The
Kubernetes version on worker nodes must be either the same version as that on the
master node, or an earlier version that is still compatible
• Image: The image to use on each node in the node pool. An image is a template of a
virtual hard drive that determines the operating system and other software for the node.
• Shape: The number of CPUs and the amount of memory allocated to each node in the
node pool.
• Subnet: One or more subnets configured to host worker nodes. The worker node
subnets must be different to the load balancer subnets.
• Quantity per Subnet: The number of worker nodes to create for the node pool in each
subnet.
• Public SSH Key: (Optional) The public key portion of the key pair you want to use for
SSH access to each node in the node pool.
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 22
Kubernetes Worker Nodes - Nodepools

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 23

Kubernetes Worker Nodes - Nodepools

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 24

Scaling Node Pools

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 25

Accessing Kubernetes Cluster using kubectl

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 26

Kubernetes Dashboard

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 27

Upgrading Kubernetes Master
• OKE Service supports in-place upgrade of Kubernetes Master nodes (via Console or
API)
• After upgrading a master node to a newer version of Kubernetes, you cannot
downgrade the master node to an earlier Kubernetes version.
• The versions of Kubernetes running on the master node and the worker nodes must
be compatible (that is, the Kubernetes version on the master node must be no more
than two minor versions ahead of the Kubernetes version on the worker nodes)

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 28

Upgrading Kubernetes Worker Nodes

• Kubernetes Worker nodes are upgraded by performing an ‘out-of-place’ upgrade.

• To upgrade the version of Kubernetes running on worker nodes in a node pool, you
replace the original node pool with a new node pool that has new worker nodes
running the appropriate Kubernetes version.
• ‘drain' existing worker nodes in the original node pool to prevent new pods starting
and to delete existing pods. Once no pods exist, the old node pool can be deleted

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 29

OCI Registry Service

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 30

Oracle Cloud Infrastructure Registry Service

• An Oracle-managed registry that enables you to simplify your development to production

workflow
• You can use OCIR as a private Docker registry for internal use, pushing and pulling
Docker images to and from the Registry using the Docker V2 API and the standard
Docker command line interface (CLI)
• You can also use OCIR as a public Docker registry, enabling any user with internet
access and knowledge of the appropriate URL to pull images
• You can access Oracle Cloud Infrastructure using the Console (a browser-based
interface) or the REST API.

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 31

Pre-requisites for OCIR

• To use registry service, user is either a part of the admin group or part of a group to which
a policy grants the appropriate permissions
• allow group acme-viewers to inspect repos in tenancy - Ability to
see a list of all repositories in Oracle Cloud Infrastructure Registry belonging to the
tenancy
• allow group acme-managers to manage repos in tenancy - Ability to
perform any operation on any repository in Oracle Cloud Infrastructure Registry that
belongs to the tenancy (Pull an image, push an image, create/delete repos etc.)
• Note: repos are tenancy-level resources, policies controlling access to them need to
go into the root compartment (i.e., the tenancy).
• User needs to have an OCI username and auth token before being able to push/pull an
image.

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 32

OCIR Repositories

• Repositories can be private or public.

• Any user with internet access and knowledge of the appropriate URL can pull images
from a public repository in Oracle Cloud Infrastructure Registry.
• To Create a Repository via Console
• Containers  Registry  Create Repository
– Repository Name
– Public or Private

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 33

Push/Pull images from OCIR

Region Region
• You use Docker CLI to push/pull images to repos in OCI Code Name
• Create a Auth Token for User and copy it phx Phoenix
• Login into OCIR iad Ashburn
• docker login <region-code>.ocir.io fra Frankfurt
– <tenancy_name>/<username> lhr London
– Auth-token
• Find images in your local repository to be pushed to OCIR and tag it appropriately in the format
• <region-code>.ocir.io/<tenancy-name>/<repos-name>/<image-name>:<tag>
• docker tag 9f1191b287da iad.ocir.io/jamalarif/testing/tomcat:1.2
• Push your tagged image to OCIR
• docker push iad.ocir.io/jamalarif/testing/tomcat
• Similarly images can be pulled using docker pull
• docker pull <region-code>.ocir.io/<tenancy-name>/<repos-name>/<image-name>:<tag>
• docker pull iad.ocir.io/jamalarif/testing/tomcat:1.2

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 34

OCIR Image Layers

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 35

Pulling images from Registry for Kubernetes Deployments

In order to pull images that reside in Oracle Cloud Infrastructure Registry

• Create a Docker registry secret, containing the Oracle Cloud Infrastructure credentials to
use when pulling the image.
• Specify the image to pull from Oracle Cloud Infrastructure Registry, including the
repository location and the Docker registry secret to use, in the application's manifest file.

• kubectl create secret docker-registry <secret-name> --docker-

server=<region-code>.ocir.io --docker-username='<tenancy-
name>/<oci-username>' --docker-password='<oci-auth-token>'
--docker-email='<email-address>'

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 36

OKE and OCIR Pricing, Packaging and Availability
• IaaS is sold under standard universal credit model
• OKE and OCIR are features of IaaS and consume IaaS resources,
with no upcharge for the feature – available in all regions PHX, IAD, FRA, LHR
Consume the Container Deployment Platform
Buy IaaS Capacity
$/OCPU, $/GB, $/Mbps

Oracle Cloud
Infrastructure

Use OKE, consume IaaS, i.e. Use OCIR, consume IaaS, i.e.
Compute, Storage, LB etc. Object Storage, Network

OCI - Container
Engine for Kubernetes OCI - Registry
Copyright
Copyright©©
2017,
2017,
Oracle
Oracle
and/or
and/or
itsits
affiliates.
affiliates.
AllAll
rights
rights
reserved.
reserved.|
Questions

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 2 - 38