GitHub – joedayz/java-on-aks: How to use Azure Kubernetes Service end-to-end for Java apps? Illustrates using a Microservice Architecture with Spring Boot, Spring Cloud and Docker –

page_type languages products description urlFragment

sample

End-to-end experience for Java apps in Azure Kubernetes Service (AKS)

java-on-aks

This guide walks you through how to deploy and manage Java apps on the Azure Kubernetes Service.

What will you experience

You will:

  • Build Piggymetrics – build a proof-of-concept application, which demonstrates
    micro service architecture pattern using Spring Boot and Spring Cloud
  • Create Mongodb and RabbitMQ on Azure
  • Create Azure Kubernetes Service and Azure Container Registry
  • Deploy Piggymetrics to Azure Kubernetes Service
  • Troubleshoot Java apps in Azure Kubernetes Service
  • Automate and rapidly deploy changes to Azure Kubernetes Service – using GitHub Actions or Azure Pipelines
  • Rapidly deploy changes to Azure Kubernetes without disruption – blue-green deployments
  • Scale out Java apps in Azure Kubernetes Service
  • Learn about next steps to production

What you will need

In order to deploy a Java Web app to cloud, you need
an Azure subscription. If you do not already have an Azure
subscription, you can activate your
MSDN subscriber benefits
or sign up for a
free Azure account.

In addition, you will need the following:

| Azure CLI
| Java 8
| Maven 3
| Git
| ACR Docker Credential Helper
| Docker
| Kubectl
| Helm
| dos2unix
|

IMPORTANT – Start Here

Clone this GitHub repo and change directory:

git clone https://github.com/Azure-Samples/java-on-aks.git

cd java-on-aks

Create MongoDB and RabbitMQ

You can create MongoDB and RabbitMQ on Azure by following steps outlined here
and capture MongoDB and RabbitMQ coordinates and credentials in
setup-env-variables-azure.sh.

Create Azure Container Registry and Azure Kubernetes Service

Create Azure Container Registry

If you have not yet setup the development environment,
make a copy of the setup environment variables bash script:

cp .scripts/setup-env-variables-azure-template.sh .scripts/setup-env-variables-azure.sh

Prep the dev environment by populating environment variables in
.scripts/setup-env-variables-azure.sh
bash script:

# ====== Piggy Metrics Azure Coordinates
export RESOURCE_GROUP=INSERT-your-resource-group-name
export REGION=westus2
export AKS_CLUSTER=INSERT-your-AKS-cluster-name
export CONTAINER_REGISTRY=INSERT-your-Azure-Container-Registry-name

## ===== Mongo DB
export MONGODB_DATABASE=INSERT-your-mongodb-database-name
export MONGODB_USER=INSERT-your-cosmosdb-account-name
export MONGODB_URI="INSERT-your-mongodb-connection-string"
export MONGODB_RESOURCE_ID=INSERT-your-mongodb-resource-id

## ===== Rabbit MQ
export RABBITMQ_RESOURCE_GROUP=INSERT-your-rabbitmq-resource-group-name
export RABBITMQ_VM_NAME=INSERT-your-rabbitmq-virtual-machine-name
export RABBITMQ_ADMIN_USERNAME=INSERT-your-rabbitmq-admin-user-name

# Rabbit MQ
export RABBITMQ_HOST=INSERT-your-rabbitmq-host-public-ip-address
export RABBITMQ_PORT=5672
export RABBITMQ_USERNAME=INSERT-your-rabbitmq-username
export RABBITMQ_PASSWORD=INSERT-your-rabbitmq-password

Then, export these environment variables from the java-on-aks directory:

source .scripts/setup-env-variables-azure.sh

Create an Azure Container Registry instance using Azure CLI:

# Create a Resource Group, if you have not created one
az group create --name $RESOURCE_GROUP 
    --location $REGION
    
# Create Azure Container Registry
az acr create --name $CONTAINER_REGISTRY 
    --resource-group $RESOURCE_GROUP 
    --sku basic --location $REGION
    
# Log into Azure Container Registry
az acr login -n $CONTAINER_REGISTRY

Create Azure Kubernetes Service

Create an Azure Kubernetes Service instance and attach the Azure Container Registry using Azure CLI:

az aks create --name $AKS_CLUSTER 
    --resource-group $RESOURCE_GROUP 
    --location $REGION 
    --attach-acr $CONTAINER_REGISTRY 
    --node-vm-size Standard_DS3_v2 
    --node-count 5

Get access credentials for the AKS cluster:

az aks get-credentials --name $AKS_CLUSTER 
    --resource-group $RESOURCE_GROUP

Auto instrument for monitoring Java apps on Azure Kubernetes Service

If you do not have an instance of Application Insights, see
how to create Application Insights.

Download Java agent (download link)
for auto instrumenting for monitoring Java apps on an Azure Kubernetes Service.

From a Linux or MacOS terminal, execute init.sh from the release.

Open the generated values.yaml file in an editor and fill up Kubernetes Cluster
target namespace, say default, and Application Insights Instrumentation Key. Like this:

namespaces: 
  - target : "default" # kubernetes namespace where the Java apps will be auto-instrumented
    iKey: "c197cf6b-WWWW-XXXX-YYYY-ZZZZZZZZZZZZ" # Instrumentation Key of the receiving Application Insights resource

Install the Java agent for auto instrumentation:

helm install ./helm-<version>.tgz -f values.yaml --generate-name

Deploy Piggymetrics to Azure Kubernetes Service

Build and push container images for micro service apps

Build Java apps, container images and push images to Azure Container Registry
using Maven and Jib:

cd config
mvn compile jib:build 
    -Djib.container.environment=CONFIG_SERVICE_PASSWORD=$CONFIG_SERVICE_PASSWORD

cd ../registry
mvn compile jib:build

cd ../gateway
mvn compile jib:build

cd ../auth-service
mvn compile jib:build

cd ../account-service
mvn compile jib:build 
    -Djib.container.environment=ACCOUNT_SERVICE_PASSWORD=$ACCOUNT_SERVICE_PASSWORD

cd ../statistics-service
mvn compile jib:build 
    -Djib.container.environment=STATISTICS_SERVICE_PASSWORD=$STATISTICS_SERVICE_PASSWORD

cd ../notification-service
mvn compile jib:build 
    -Djib.container.environment=NOTIFICATION_SERVICE_PASSWORD=$NOTIFICATION_SERVICE_PASSWORD

Jib is configured to start from a production-ready distribution of Java. For example:

<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>jib-maven-plugin</artifactId>
    <version>1.8.0</version>
    <configuration>
        <from>
            <!-- production-ready distribution of Java -->
            <image>mcr.microsoft.com/java/jre-headless:8u232-zulu-alpine</image>
        </from>
        <to>
            <image>$CONTAINER_REGISTRY.azurecr.io/$parent.artifactId-$project.name</image>
        </to>
        <container>
            <jvmFlags>
                <jvmFlag>-Xms2048m</jvmFlag>
                <jvmFlag>-Xmx2048m</jvmFlag>
            </jvmFlags>
            <ports>
                <port>$CONFIG_PORT</port>
            </ports>
            <labels>
                <key1>$project.version</key1>
            </labels>
        </container>
    </configuration>
</plugin>

Prepare Kubernetes manifest files

Prepare Kubernetes manifest files using the supplied bash script:

# cd to kubernetes folder
cd ../kubernetes
source ../.scripts/prepare-kubernetes-manifest-files.sh

Create Secrets in Kubernetes

You can create Secrets for deploying micro service apps in Kubernetes:

kubectl apply -f deploy/0-secrets.yaml

# you can view Secrets in Kubernetes using:
kubectl get secret piggymetrics -o yaml

Deploy Spring Cloud Config Server

You can deploy the Spring Cloud Config Server to Kubernetes:

kubectl apply -f deploy/1-config.yaml

Deploy Spring Cloud Service Registry

You can deploy the Spring Cloud Service Registry to Kubernetes:

kubectl apply -f deploy/2-registry.yaml

You can validate that a Spring Cloud Config Server is up and running by
invoking its REST API.

The Spring Cloud Config Server REST API has resources in the following form:

/application/profile[/label]
/application-profile.yml
/label/application-profile.yml
/application-profile.properties
/label/application-profile.properties

You can get IP addresses of Spring Cloud Config Server and Spring Cloud Service Registry
using kubectl:

kubectl get services
NAME         TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)          AGE
config       LoadBalancer   10.0.34.171   51.143.50.77    8888:31892/TCP   3m15s
kubernetes   ClusterIP      10.0.0.1      <none>          443/TCP          3h42m
registry     LoadBalancer   10.0.198.13   52.137.96.211   8761:31689/TCP   6s

Try:

open http://<EXTERNAL-IP-of-config>:8888/gateway/profile
open http://<EXTERNAL-IP-of-config>:8888/account-service/profile
open http://<EXTERNAL-IP-of-config>:8888/statistics-service/profile
open http://<EXTERNAL-IP-of-config>:8888/notification-service/profile
...
open http://<EXTERNAL-IP-of-config>:8888/notification-service/profile/development
...

You can validate that a Spring Cloud Service Registry is up and running by
opening the Service Registry Dashboard:

open http://<EXTERNAL-IP-of-registry>:8761/

Deploy Spring Cloud Gateway

You can deploy the Spring Cloud Gateway to Kubernetes:

kubectl apply -f deploy/3-gateway.yaml

Deploy 4 Spring Cloud micro service apps

You can deploy Spring Cloud micro service apps to Kubernetes:

kubectl apply -f deploy/4-auth-service.yaml
kubectl apply -f deploy/5-account-service.yaml
kubectl apply -f deploy/6-statistics-service.yaml
kubectl apply -f deploy/7-notification-service.yaml

You can validate that Spring Cloud middleware components and micro service apps are running:

kubectl get services
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)          AGE
account-service        ClusterIP      10.0.215.174   <none>          6000/TCP         108s
auth-service           ClusterIP      10.0.164.45    <none>          5000/TCP         2m26s
config                 LoadBalancer   10.0.34.171    51.143.50.77    8888:31892/TCP   24m
gateway                LoadBalancer   10.0.154.13    52.143.88.12    80:31412/TCP     4m3s
kubernetes             ClusterIP      10.0.0.1       <none>          443/TCP          4h3m
notification-service   ClusterIP      10.0.169.231   <none>          8000/TCP         63s
registry               LoadBalancer   10.0.198.13    52.137.96.211   8761:31689/TCP   21m
statistics-service     ClusterIP      10.0.217.229   <none>          7000/TCP         76s

You can also validate that by
opening the Spring Cloud Service Registry Dashboard

open http://<EXTERNAL-IP-of-registry>:8761/

Open Spring Cloud micro service apps running on Kubernetes

Open the Piggymetrics landing page by using thegateway app’s EXTERNAL-IP.

open http://<EXTERNAL-IP-of-gateway>/

For example:

Troubleshooting micro service apps in Azure Kubernetes Service

With out-of-the-box support for aggregating logs, metrics, and
distributed app traces into Azure Monitor, you can easily visualize
how your applications are performing, detect and diagnose issues
across micro service applications and their dependencies, drill
into monitoring data for troubleshooting and gain better
understanding of what end-users do with your apps.

Debug in development machine

You can run Spring Cloud Config, Spring Cloud Service Registry,
Spring Cloud Gateway and other Spring Cloud components on their dev machine.
You can attach debuggers to Spring Cloud micro service apps and step through them. You can
look at logs and metrics. Use Java Flight Recorder, etc.

Stream logs from micro service apps in cloud to development machines

You can stream logs from micro service apps running on Kubernetes to your
development machine using kubectl, like:

# Stream logs from Spring Cloud Config Server
kubectl logs -f --timestamps=true -l app=config

# Stream logs from Spring Cloud Service Registry
kubectl logs -f --timestamps=true -l app=registry

# Stream logs from Spring Cloud Gateway
kubectl logs -f --timestamps=true -l app=gateway

# Stream logs from Spring Cloud micro service apps
kubectl logs -f --timestamps=true -l app=auth-service
kubectl logs -f --timestamps=true -l app=account-service
kubectl logs -f --timestamps=true -l app=statistics-service
kubectl logs -f --timestamps=true -l app=notification-service

Use aggregated logs and metrics in Azure Log Analytics

You can aggregate logs in Azure Log Analytics and retrieve them
using Kusto queries. If you do not have a Log Analytics Workspace in Azure,
see how to create a Log Analytics Workspace

You can onboard your Kubernetes cluster to Azure Monitor for monitoring, by clicking
on the Logs blade in the Azure Portal and choosing your Log Analytics Workspace:

Then, you can view logs using Kusto queries in the Logs blade of your
Log Analytics Workspace:

Here are some sample Kusto queries for viewing logs for each of the micro service apps –
please replace java-on-aks with your Azure Kubernetes cluster name:

-- Logs for Spring Cloud Config Server
let ContainerIdList = KubePodInventory
| where ContainerName contains 'config'
| where ClusterId contains 'java-on-aks'
| distinct ContainerID;
ContainerLog
| where ContainerID in (ContainerIdList)
| where LogEntry !contains "AI:" 
| project LogEntrySource, LogEntry, TimeGenerated, Computer, Image, Name, ContainerID
| order by TimeGenerated desc
| render table


-- Logs for Spring Cloud Service Registry
let ContainerIdList = KubePodInventory
| where ContainerName contains 'registry'
| where ClusterId contains 'java-on-aks'
| distinct ContainerID;
ContainerLog
| where ContainerID in (ContainerIdList)
| where LogEntry !contains "AI:" 
| project LogEntrySource, LogEntry, TimeGenerated, Computer, Image, Name, ContainerID
| order by TimeGenerated desc
| render table


-- Logs for Spring Cloud Gateway
let ContainerIdList = KubePodInventory
| where ContainerName contains 'gateway'
| where ClusterId contains 'java-on-aks'
| distinct ContainerID;
ContainerLog
| where ContainerID in (ContainerIdList)
| where LogEntry !contains "AI:" 
| project LogEntrySource, LogEntry, TimeGenerated, Computer, Image, Name, ContainerID
| order by TimeGenerated desc
| render table


-- Logs for Account Service micro service app
let ContainerIdList = KubePodInventory
| where ContainerName contains 'account-service'
| where ClusterId contains 'java-on-aks'
| distinct ContainerID;
ContainerLog
| where ContainerID in (ContainerIdList)
| where LogEntry !contains "AI:" 
| project LogEntrySource, LogEntry, TimeGenerated, Computer, Image, Name, ContainerID
| order by TimeGenerated desc
| render table


-- Logs for Auth Service micro service app
let ContainerIdList = KubePodInventory
| where ContainerName contains 'auth-service'
| where ClusterId contains 'java-on-aks'
| distinct ContainerID;
ContainerLog
| where ContainerID in (ContainerIdList)
| where LogEntry !contains "AI:" 
| project LogEntrySource, LogEntry, TimeGenerated, Computer, Image, Name, ContainerID
| order by TimeGenerated desc
| render table

Use Application Insights to monitor your micro service apps

You can use Application Insights to monitor your live web application. It will
automatically detect performance anomalies. It includes powerful analytics tools to
help you diagnose issues and to understand what users actually do with your app. It is designed to
help you continuously improve performance and usability.

After some time, you can see distributed tracing in the
configured Application Insights instance. Go to the `Application Map’ blade in the Azure Portal:

Also, you can view the performance and call drill downs in the Performance blade:

Automate and rapidly deploy changes to Azure Kubernetes Service – GitHub Actions or Azure Pipelines

Create an Azure Pipelines CI/CD pipeline that automatically builds the code and deploys it to the Azure Kubernetes Cluster whenever there’s a commit to the repository.

Prerequisites

There are some additional prerequisites for this automation:

[!Important]
To simplify the service connection, use the same email address for Azure DevOps as you use for Azure.

  • Create an Azure KeyVault and upload secrets.

AZURE-REGISTRY-USERNAME=00000000-0000-0000-0000-000000000000
AZURE-REGISTRY-PASSWORD=<Token obtained according to instrucciones in it url: https://docs.microsoft.com/en-us/azure/container-registry/container-registry-authentication>

Ensure the service principal used in the service connection above has GET, LIST permissions on the vault. Use below command for that:

$KV_NAME = kv_java_demo
<clientId from the Azure SPN JSON> is obtained from the Azure Resource Manager created earlier.

Next, you must to copy the cliente Id.

az keyvault set-policy -n $KV_NAME --secret-permissions get list --spn <clientId from the Azure SPN JSON>

Azure Pipelines

  1. Sign into Azure Pipelines and Create a Pipeline using the azure-pipelines.yml file.

  2. Take a look at the pipeline to see what it does. Make sure that all the default inputs are appropriate for your code.

    The azure-pipelines.yml file contains the following key elements:

    • The trigger at the top indicates the commits that trigger the pipeline, such as commits to the master branch.
    • The variables which parameterize the YAML template
    • The stages
      • Build stage, which builds your app, and a Deploy stage, which deploys it to AKS cluster.
      • Deploy stage also refers the Environment with Kubernetes resource. Ensure to modify the environment name to the one that you have created.
    • AzureKeyVault task is used in both the stages to fetch the secrets from Azure Key Vault instance and set as variables. In the Deploy stage, these variables are used to set secrets in the pods.
    • Kubernetes Manifest task has the added benefits of being able to check for object stability before marking a task as success/failure, perform artifact substitution, add pipeline traceability-related annotations onto deployed objects, simplify creation and referencing of imagePullSecrets

  3. After you’ve looked at what the pipeline does, select Save and run, after which you’re prompted for a commit message because Azure Pipelines adds the azure-pipelines.yml file to your repository. After editing the message, select Save and run again to see your pipeline in action.

  4. As your pipeline runs, watch as your build stage, and then your deployment stage, go from blue (running) to green (completed). You can select the stages and jobs to watch your pipeline in action.

  5. Additionally, you can also explore Kubernetes objects created and the deployment history for the App by navigating to the “Environment“.

    • From the pipeline summary -> Select the Environments tab -> Select View environment
    • Resources view within the environment provides a glimpse of the status of objects within the namespace mapped to the resource. It also overlays pipeline traceability on top of these objects so that one can trace back from a Kubernetes object to the pipeline and then back to the commit.

GitHub Actions

You can also configure a
Workflow to automate build and deploy
in GitHub
using the workflow.yml.

  • Create secrets for every parameter in .scripts/setup-env-variables-azure-template.sh.

    • Set GitHub Repo Secrets AZURE_CREDENTIALS, AKS_CLUSTER_NAME, AKS_RESOURCE_GROUP and AKS_NAMESPACE with Azure creds, AKS cluster name, resource group name and namespace
    • For configuring App secrets, You could either set them as GitHub Secrets or fetch them from an Azure Key Vault instance.
  • workflow.yml contains the following key elements:

    • The on: [push] at the top indicates the commits that trigger the pipeline, such as every commit to the branch.
    • The env: with variables which parameterize the YAML template
    • The jobs : Build job, which builds your app, and a Deploy job, which deploys it to AKS cluster. Each job has steps which either uses an action or run a script, all of which execute on a hosted-runner defined by runs-on.
    • AzureKeyVault action is used in both the jobs to fetch the secrets from Azure Key Vault instance and set as environment variables. In the Deploy job, these variables are used to set secrets in the pods.
    • Actions to deploy to AKS are then used to create imagepullsecret, set secrets on pods and finally to deploy to AKS cluster.

Rapidly deploy changes to Azure Kubernetes Service without disruption – blue-green deployments

Scale out micro service apps in Azure Kubernetes Service

You can scale out micro service apps in Azure Kubernetes Service:

kubectl scale deployment gateway --replicas=4

kubectl describe deployment gateway
Name:                   gateway
Namespace:              default
CreationTimestamp:      Sat, 07 Dec 2019 22:36:27 -0800
Labels:                 app=gateway
                        project=piggymetrics
                        tier=frontend
Annotations:            deployment.kubernetes.io/revision: 1
                        kubectl.kubernetes.io/last-applied-configuration:
                          {"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":,"labels":{"app":"gateway","project":"piggymetrics","ti...
Selector:               app=gateway,project=piggymetrics,tier=frontend
Replicas:               4 desired | 4 updated | 4 total | 4 available | 0 unavailable

Congratulations and Next Steps to Production

Congratulations!!

You built, deployed, scaled out and setup monitoring for Spring Cloud micro service apps
using Spring Boot and Spring Cloud, Azure Kubernetes Service, Azure Container Registry,
Azure Monitor, Log Analytics and Application Insights.

To advance Java deployments on Kubernetes to production, you should think
about the following next steps:

  • Frame a scalable infrastructure with Azure Kubernetes Service and Azure Container Registry
    to satisfy your deployments’ business and technical requirements. See Microservices architecture on Azure Kubernetes Service (AKS)
  • Frame a plan to assume infrastructure management — including load balancers, clustering,
    security, SSL offloading, domain name mapping, etc.
  • Define an application lifecyle for apps on Azure Kubernetes Service and use them
    across developer and DevOps automation tools. This lifecyle should include a
    source-to-container strategy
  • Frame a plan to assume service management — including configuration, managing secrets, backup & restore,
    monitoring & log shipping, upgrades, auto-restart, auto-scale or manual-scale, etc. If you are deploying
    Spring Cloud micro service apps, service management should include
    strategies for manually or dynamically scaling Spring Cloud middleware components –
    Spring Cloud Config Server,
    Spring Cloud Service Registry, Spring Cloud Gateway, etc.

Resources

Credits

This Java micro services sample is forked from
sqshq/Piggymetrics – see Piggymetrics README.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct.
For more information see the Code of Conduct FAQ or
contact [email protected] with any additional questions or comments.