Search

myjavabytes

My technical short notes

Getting a host path persistent volume to work in minikube (kubernetes run locally)

Problem

I have had a hard time figuring out how to make a persistent volume work in minikube running on Mac OS.

Solution

I could get this to work only if I used the xhyve driver when you create minikube. How do you do that?

Step 1: Install xhyve driver

https://github.com/kubernetes/minikube/blob/master/DRIVERS.md#xhyve-driver

$ brew install docker-machine-driver-xhyve

# docker-machine-driver-xhyve need root owner and uid
$ sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
$ sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve

Step 2: Create and start minikube using xhyve driver

minikube start --vm-driver=xhyve

Step 3: Verify that your /Users folder from Mac is auto-mounted in the minikube vm

minikube runs a VM. The /Users folder from your Mac gets mounted on this VM at the same location /Users by default. You can verify this by going into the minikube vm.

$ minikube ssh
$ cd /Users
$ ls

Step 4: Create a Persistent Volume in kubernetes using hostPath

When you create a persistent volume in kubernetes using a hostPath, the hostPath can use any location under /Users. Here is a sample YAML file for creating a persistent volume in kubernetes:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: microxs-commerce-jenkins-volume-1
spec:
  accessModes:
    - ReadWriteOnce
  capacity:
    storage: 5Gi
  hostPath:
    path: /Users/lijojacob/bitbucket/microxs-commerce/jenkins-data

Step 5: Use the persistent volume in a persistent volume claim

Pods can use persistent volume claims. The persistent volume claim can in-turn use the persistent volume we created above. Here is the YAML file for a persistent volume claim:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: microxs-commerce-jenkins-volume-1-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeName: microxs-commerce-jenkins-volume-1

Step 6: Go ahead and use your persistent volume claim in your pod/deployment

Here is a YAML snippet of using the persistent volume claim in your pod

volumes:
  - name: microxs-commerce-jenkins-volume-1
    persistentVolumeClaim:
      claimName: microxs-commerce-jenkins-volume-1-claim

Step 5: Externalized configuration in a distributed system using spring-cloud-config

Objective

A lot of our microservice configuration now resides in the yaml configuration files within the application. In this episode, we will make this configuration environment-specific (12-factor-app), and externalize this configuration by serving it from a configuration service. We use spring-cloud-config server and client to achieve this.

At the end of this episode, based on the environment in which you run your application (eg: local, QA, cloud), the application will pull its configuration (eg: database connection properties) from a configuration service, which in turn gets all the configuration from a git repository.

Technologies introduced

  1. Spring Cloud Config

Create a repository to hold your configuration

  • Here I have gone for a single repository to hold the configuration of all my services. We could have gone for “one repo per application” policy also.
  • Here is my repository:
https://github.com/microxs-demo/application-configuration.git
  • In this repository, I have configuration yaml files for each of my spring applications.
  • The different environments (eg: development, QA, Prod) are represented as spring profiles in each of these yaml files.

Create a configuration service

  • Next, we create a configuration service that serves the configuration from the above git repository
  • It is a spring boot application with its main class annotated with
    @EnableConfigServer
  • Note that the config-service application depends on
    org.springframework.cloud:spring-cloud-config-server:1.2.2.RELEASE
  • Specify the git repository from which the config-service should serve configuration
    spring.cloud.config.server.git.uri: https://github.com/microxs-demo/application-configuration.git
  • Run your config-service

Verify the configurations served by config-service

  • In your browser, go to:
    http://localhost:8888/catalog-service/development
  • The configuration for catalog-service for the profile – “development” will be served in JSON format

Update your services to use configuration from config-service

We will now update our catalog-service to use configuration from the externalized configuration server – config-service.

  • Add dependency to
    org.springframework.cloud:spring-cloud-starter-config:1.2.2.RELEASE
  • Set the config-service URI for the application to fetch its configuration from. Update the configuration file catalog-service.yml
spring:
  profiles: development
  cloud:
    config:
      uri: http://${MICROXS_COMMERCE_CONFIG_SERVICE_HOST:localhost}:${MICROXS_COMMERCE_CONFIG_SERVICE_PORT:8888}

Run your service choosing the profile (or environment) you want

SPRING_PROFILES_ACTIVE=development gradle bootRun

Your logs should indicate the profile you are running, show where it is pulling configuration from:

Fetching config from server at: http://localhost:8888

Located environment: name=catalog-service, profiles=[development], label=master, version=d7ca69a6470a4e72826e5bd59619ca1c667eb503, state=null

Located property source: CompositePropertySource [name='configService', propertySources=[MapPropertySource [name='configClient'], MapPropertySource [name='https://github.com/microxs-demo/application-configuration.git/catalog-service.yml#development'], MapPropertySource [name='https://github.com/microxs-demo/application-configuration.git/catalog-service.yml']]]

The following profiles are active: development

You will also notice that your mongodb connection properties are coming from whatever you defined for that profile in the configuration repository.

Source Code

Tag “step5” in repository: application-configuration

git clone https://github.com/microxs-demo/application-configuration 
cd application-configuration 
git checkout tags/step5

Tag “step5” in repository config-service

git clone https://github.com/microxs-demo/config-service 
cd config-service 
git checkout tags/step5

Tag “step5” in repository catalog-service

git clone https://github.com/microxs-demo/catalog-service 
cd catalog-service 
git checkout tags/step5

Video

Detailed explanation in video

Step 4: Run our first docker container

Objective

We built our first microservice and made a docker image of it in previous episodes. Here we run our docker container.

Technologies

Docker

Create a new network

Out catalog-service needs to be able to connect to mongo. For that they need to be on the same network. So, let’s create a network, to which we will attach our containers later.

docker network create microxs-demo-network

Run mongodb container

Our microservice – catalog-service needs a mongodb. Hence, we will first run mongodb as a container. MongoDB’s docker image is available on docker hub. Pull it and run.

docker run --net=microxs-demo-network --name catalog-db mongo

This should pull the image of mongo from docker hub, create a container named catalog-db from the image, and then run it. It also attaches the container to the network – “microxs-demo-network”. It should give the following output to indicate that it is running and listening on the default mongo port

[thread1] waiting for connections on port 27017

Run catalog-service container

docker run --net=microxs-demo-network --name=catalog-service --env MICROXS_COMMERCE_MONGO_SERVICE_HOST=catalog-db --env MICROXS_COMMERCE_MONGO_SERVICE_PORT=27017 -p 8080:3333 catalog-service

This command pulls the docker image “catalog-service” and runs it.

Let’s dissect this command: –

--net=microxs-demo-network

Join the network “microxs-demo-network”. Host name will be “catalog-service”.

--name=catalog-service

It gives the container the name as “catalog-service”.

--env MICROXS_COMMERCE_MONGO_SERVICE_HOST=catalog-db

Set the environment variable MICROXS_COMMERCE_MONGO_SERVICE_HOST to “catalog-db” which is the hostname of the mongo db server. This environment variable is used in catalog-service to generate connection URL to mongo.

--env MICROXS_COMMERCE_MONGO_SERVICE_PORT=27017

Similarly set the environment variable for mongodb port.

-p 8080:3333

Expose the container’s port 3333 to port 8080 on the host machine.

You can verify this by running the command:

docker ps

Also, invoke the spring-data-REST endpoint that we created in catalog-service.

curl http://localhost:8080/catalog-service/skus

Video

Step 4 Video

Step 3: Dockerize our first microservice

Objective

Make a docker image of our first microservice – the catalog-service. Create a Dockerfile, and build a docker image out of it.

Technologies introduced

  1. Docker
  2. Dockerfile

Create a Dockerfile describing the image

What does your Dockerfile contain? See the complete Dockerfile.

  • Which is the base image for this container? What do we need for our application to run?
    1. An operating system
    2. A JRE

The base image I have chosen here is openjdk:8u92-jre-alpine, which is an open JDK installed on an alpine LINUX system. Alpine linux is a small, efficient LINUX distribution.

FROM openjdk:8u92-jre-alpine
  • Work Directory
WORKDIR /catalog-service

Within the container, the application will run from this folder

  • Version of app as argument
ARG version

This is not absolutely essential, but here we are taking the “version” of catalog-service as an argument to the Dockerfile, so that we can choose to use any version to build the image

  • Adding the catalog-service JAR file to the image
ADD ./build/libs/catalog-service-${version:-1.0.0.RELEASE}.jar /catalog-service/catalog-service.jar

We are copying the specified version (1.0.0.RELEASE, if no version argument provided) of catalog-service from our build directory into the work directory /catalog-service

  • What command should be run when the container starts?
CMD java -jar catalog-service.jar

We ask docker to run the java command and execute the Main class from the catalog-service JAR.

Build an image from the Dockerfile

Change directory into the folder that contains the Dockerfile, and do a docker build.

cd catalog-service
docker build --tag=catalog-service .

Here we name the image as “catalog-service” using the –tag argument.

Verify that the image was created

List the docker images in your machine and verify that catalog-service is listed there:

docker images

Source code

Tag “step3” in repository “catalog-service

git clone https://github.com/microxs-demo/catalog-service
cd catalog-service
git checkout tags/step3

Video

Step 3 video

 

 

Step 2: Spring Boot, MongoDB, Data REST services

Objective

Add a database to store the catalog. Expose the repository as RESTful webservices.

Technologies used

  1. Spring Boot
  2. MongoDB
  3. spring-data-mongodb
  4. spring-data-rest

MongoDB integration

  • I have chosen MongoDB as the database for our catalog microservice.
  • spring-data-mongodb module helps us integrate our application with mongodb by providing us with a POJO based entity model, and also allowing us to write repository style data access layer easily. The new dependency we introduced to our project is: –
org.springframework.boot:spring-boot-starter-data-mongodb
  • We created our catalog service’s entities which map to mongo db documents. They are annotated with @Document annotation.
  • MongoDB connection properties are configured in catalog-service.yml
  •  We also created repositories (MongoRepository) for all entities that we want to expose.
  • Enable Mongo Repositories using annotation: –
    @EnableMongoRepositories

Expose repositories as RESTful services

  • Next, we expose our repositories as RESTful services using spring-data-rest module. For this, we included the dependency: –
org.springframework.boot:spring-boot-starter-data-rest
  • Base Path for the RESTful services are also configured in catalog-service.yml

Source Code

Tag “step2” in repository: catalog-service

git clone https://github.com/microxs-demo/catalog-service
cd catalog-service
git checkout tags/step2

Video

Detailed explanation in video

 

Step 1: Web app using spring-boot

Objective

This is the first step towards building our first microservice – the catalog-service. In this step we will create a web application using spring-boot framework and run it.

Technologies used

  1. Java
  2. Spring boot
  3. gradle

Create the web application

  • Create a web application using gradle. Add a build.gradle file with the required dependencies:
compile("org.springframework.boot:spring-boot-starter-web")

The “spring-boot-starter-web” dependency does all the auto-configuration for a web application. It includes a tomcat container.

  • The buildscript also depends on the plugin: –
classpath("org.springframework.boot:spring-boot-gradle-plugin:1.5.1.RELEASE")

This plugin gives the ability to launch bootRun from gradle (to run a spring boot application)

  • Create a Main class – CatalogServer annotated to identify it as a Spring Boot app.
@SpringBootApplication

The main method will start the Spring Boot application.

SpringApplication.run(CatalogServer.class, args);

Build the web application

gradle build

Run the spring boot application

gradle bootRun

Source Code

Tag: – “step1” in repository: – “catalog-service

git clone https://github.com/microxs-demo/catalog-service
cd catalog-service
git checkout tags/step1

Video

Step 1 video

Building an e-commerce app using microservices

This is a multi-part series on building an ecommerce application using microservices.

An ecommerce application is built of numerous functional modules like catalog, shopping cart, checkout, payment, order management, fulfillment, inventory to name a few.

Typically ecommerce applications are built as huge monolithic applications. This is an attempt to build the same using microservices. The merits and de-merits of using microservices has been dicussed at large in various forums. Whether microservices architecture is right for your organization is also a different topic of discussion. Here we assume you made that choice, and we see how we can build one.

I have made some choices in languages used and technology stack. Here are some of those, and we will discover more as we go further: –

  1. Java
  2. Spring framework
  3. Spring Boot
  4. MongoDB
  5. Docker
  6. Kubernetes
  7. Angular

We will follow a step by step process of building each microservice, and then integrating them.

Each step will be supplemented by videos, and the code for each step will be available in github repositories.

Here is my youtube channel where you can find the videos: – http://www.youtube.com/c/LijoJacobPankavil

Here is the github organization under which all the code will be available: – https://github.com/microxs-demo

Feel free to leave your opinion in comments against each blog.

 

 

 

Mount hostPath persistent volume in minikube on MacOSX

If you are trying to mount a host path as a persistent volume in minikube, and you are running minikube on MacOSX, you are likely to be faced with permission denied issues when using the persistent volume.

Here is the steps to overcome this:-

echo "/Users -network 192.168.99.0 -mask 255.255.255.0 -alldirs -maproot=root:wheel" | sudo tee -a /etc/exports
sudo nfsd restart
minikube start
minikube ssh -- sudo umount /Users
minikube ssh -- sudo /usr/local/etc/init.d/nfs-client start
minikube ssh -- sudo mount 192.168.99.1:/Users /Users -o rw,async,noatime,rsize=32768,wsize=32768,proto=tcp

Source: Minikube Github Issue

 

 

12 factor apps

As developers, we all might have followed all or some of these at some point, but today it is important that our apps (web applications, or any other) follow all of these. They are the 12 factors. Read through this checklist:- https://12factor.net/

Create a free website or blog at WordPress.com.

Up ↑