Category Archives: dev

Build Java Container Images in seconds using Quarkus CLI

Posted on by
Reply

Here's a quick 7 minute video I threw together that shows you how to use the Quarkus CLI to build images using Podman / Docker and Jib and runs the container using Podman Desktop.

TL;DR If you just want to try it out, just run this:
quarkus image build

—
As a Java developer, getting started with building containers can get complicated real quick. At the face of it you just copy your artifact into a base image and do a docker or podman build and that's it right?! On a serious note, creating a container from your application requires quite a few more considerations than that. To begin with, you have to start with a base image: do you start from scratch, or use a base image that already has a jvm implementation included? Or maybe you've opted to do a native build of your application and you don't need a jvm running in your container. Either way, you're going to need to find a trusted source that provides tested, secured, possibly signed, and stable images for the particular Java version you're targetting. And hopefully this source will be able to support you as well as being able to maintain and backport patches going forward. I'm of course a little bit biased, but I always start from Red Hat's Universal Base Images (UBI). These UBIs are free to use without restrictions which is pretty cool, and they are at the core a subset of the Red Hat Linux (RHEL) distribution so you know they're going to be stable and trusted. There are several JVM based UBI images, as well as micro UBIs that can be used if you're going to deploy a native binary.

Once you have decided on a base image, you will need a way to build your container. Docker build is the most known way of building containers, but certainly not the only one. You can use Podman, a pure open source alternative with more security capabilities built in, or you can use BuildPacks or Jib, the latter being a container build tool specifically created for building container images with Java. The downside is that for each of these tools, you need to know how to build a container image; and moreover, you'll need to have knowledge of the above mentioned base images and how to write a Dockerfile/Containerfile to copy your artifacts and dependencies into them.

Quarkus & Containers

Quarkus wants to bring Developer Joy and it is not different when it comes to building containers. When you create a Quarkus project, whether you use Quarkus CLI quarkus create app, code.quarkus.io or one of the quarkus plugins in VS Code or IntelliJ, Quarkus will create a folder in your src/main called 'docker'. And in this src/main/docker, you'll find a few prebaked Dockerfiles that are ready to go using those UBI base images I mentioned before. What's even better is that the Quarkus plugin has knowledge of these Dockerfiles so you can build images using your favorite project build tool (Maven or Gradle). Basically, you can do an image build with eg. mvn quarkus:image-build (make sure to mvn compile your code first) and quarkus will automatically build a container image using, by default, the Dockerfile.jvm that came with your Quarkus project. Pretty sweet right? But it gets better.

The Quarkus CLI

The Quarkus CLI is basically 'just' a convenience wrapper around Maven or Gradle (depending on your preference) that lets you create projects, manage extensions and do essential build and dev commands. This is where things get interesting, because you can do really simple commands that are easy to remember such as quarkus image build. Behind the scenes, Quarkus will invoke your build tool (by default Maven, but can be Gradle as well) and run both `mvn compile` and `mvn quarkus:image-build`. So you can literally compile your code and build the container image with the opinionated base images provided by Quarkus with one simple command. That's pretty cool if you ask me. Actually, it could get even better because if you want to push your built container to a registry as well, it could be done in one go with the artifact and container build like so quarkus image push --also-build!

What if I don’t have Docker or Podman on my machine?

Quarkus has extensions for other container build tools such as Jib or Buildpacks. You just have to add the extension, eg quarkus ext add jib and subsequently specify your build strategy when you invoke the quarkus build command: quarkus image build jib.

Podman Desktop

Another alternative is to use Podman Desktop. This tool is FOSS (Free Open Source Software) and makes managing containers really easy. When you install it, it'll help you install the underlying Podman engine. Once installed, you can spin up your container using the Podman desktop UI. Just go to the Images tab and hit the play button next to your container image. Podman will automatically configure the port the app will run on (usually 8080 by default in Quarkus) but it’ll also automatically figure out if that port is already being used and reassign the container to use a different port instead.

podman desktop view images
Podman Desktop's "Images" view, where you can see the created container image and the play button to start it

podman desktop start container from image view
Podman Desktop's "Create Container" view, where you can define settings and start the container

podman desktop view containers
Podman Desktop's "Containers" view listing the containers on your machine

podman desktop view container logs
Log output from the running container

I hope you liked this post. If you have any questions or comments, feel free to share them in the comments section. And don't forget, sharing is caring :)

Using Kafka with Camel Quarkus

Posted on by
Reply

I recently spoke at Devoxx Belgium about Camel Quarkus (you can find the recording here: https://youtu.be/FBWgbhp8FG8) and for this talk I wanted to demonstrate just how easy it is to integrate different components using Camel. I also wanted some interaction with the audience and make the session a little more fun by having them participate in the demo.

My initial idea was a voting app where the audience could vote for their favorite IDE by using Twitter. It seemed like a neat idea, I could use the Twitter component of Apache Camel which I had already demo'd during a talk I did for the Openshift Coffee Hour (https://youtu.be/aaEV19c6GhQ), but ultimately I decided against it because I really wanted more than just a handful of votes (if that). Since my talk was also about using Camel Quarkus in a Serverless way I needed a LOT of votes from the audience :P.

What I came up with was to build a small web app where the audience could vote through the UI which would send a REST Get request to an API that I would expose using Camel; and then update a database that I could poll to show the results live on the big screen. I didn't want the API requests to immediately update the database, but instead stream them to a Kafka topic first and then handle the requests and update the database under a controlled load. That would allow me to also demonstrate how easy it is to integrate Camel with Kafka. It's basically one line of code:

to("kafka:{{kafka.topic.name}}")

Easy right?

I still wanted to show the Twitter component as well as an additional feature, and finally, just in case there weren't going to be too many votes, I prepared another rest endpoint that I could call with jmeter to 'cheat' the result so I could show the audience how the system would scale pods out rapidly under load. Using Quarkus and Knative that's super easy to do since Quarkus starts up super fast, and Knative scales out rapidly based on requests.

Architecture

The final architecture looks a little bit like this (forgive my crappy architecture design skills)

Camel Quarkus Voter Architecture

The UI application features a, well, UI, that shows the results of a poll and a form to vote for your favorite Java stack. When you vote, a REST POST event gets sent to the 'ingester' app, which will translate the result and add it to a Kafka topic.
The 3rd application (processor) consumes the Kafka messages at its own pace and updates the database accordingly. (eg. if someone voted 'Vim', the counter for vim would be incremented by 1) The 'retriever' application (currently embedded in the processor app) has a REST GET endpoint to get the results from the DB.

Anatomy

So ultimately the application is composed of the following components:

UI
The ui application displays a list of java stacks/frameworks that you can vote for by clicking the respective button next to it. This action calls the ingester app. The page also displays a bar chart of the results so far. The app is built with Quarkus Qute templating and some crappy javascript/jquery code :P

Ingester
The ingester Camel Quarkus application receives requests from the user (via HTTP) and forwards the requests to the Kafka broker. The main component of the application:

  • RestToCamelRoute : Camel route that receives a rest call and forwards it on to a Kafka broker

Processor
The processor Camel Quarkus application receives the vote requests from Kafka, processes them, and writes results into the votesdb Postgres DB table. As of right now it also returns the results through a /getresults endpoint. The application has the following Camel Routes:

  • processor/VotesRoute consumes messages from a kafka topic (in json format), extracts the value of 'shortname' and increments the counter of the java stack that matches with this shortname.
  • RestRoute returns data from the votes table in json format

Twitter
The twitter Camel Quarkus application polls twitter for matches to my handle (@kevindubois) and scans for matches to the various IDEs I'm using.

  • TwitterRoute polls twitter, and forwards any matches to the same Kafka topic messages

And this is basically the UI of the voting app:

You can find the source code in my github: https://github.com/kdubois/CamelQuarkusVoter

Using Camel Quarkus to send alerts on Telegram when web content changes

Posted on by
Reply

Ahh… Covid-19. From thinking it was just another little outbreak in Asia to months long lockdowns and seeing friends and family succumb to this nasty little virus, it's been a surreal year-and-a-half. Earlier this year we finally got some hope with the rollout of vaccines, and I personally was very eager to get vaccinated and hopefully leave my anxiety to leave the house behind me. I currently reside in Brussels, Belgium, and unlike in the US, the vaccine rollout was, at least initially, painfully slow. The Brussels government did however set up a nice little web application where they would keep everyone up to date with the current age group that was eligible to get vaccinated. The only caveat was that there was no way to keep informed of any changes to the age group other than refreshing the website. Which I did several times a day for a day or two. But, I wouldn't work in tech if it didn't seem a bit silly to perform a manual repetitive task right? :D

I saw a while ago a nice little demo of one of my colleagues at Red Hat who used Red Hat Fuse (based on Apache Camel) to send a message to Telegram, and this seemed like a pretty good way to go since I use Telegram on a daily basis anyway. The advantage of using an Enterprise Integration Pattern based system is that it makes the solution quite flexible, so if at some point I want to send the message to another endpoint then all I need to change is the "to" parameter of my Camel route and I'm good to go. Likewise, if I want to adapt this code to scrape bits of data from another website, then that's a very easy change as well.

Anyway, take a look at the video and let me know what you think!

The source code can be found in my Github: https://github.com/kdubois/datascraper. Feel free to scrutinize and send pull requests with improvements!

Synchronize between Subversion and Git using Continuous Integration Pipelines and Containers

Posted on by
Reply

I recently worked on a project for a client that (still) uses Subversion for their code repositories. Unfortunately they had no real short-term interested in migrating their code to Git, which left the team without such goodies as merge/pull requests, easy branching and all the other advantages that Git provides. Git actually has a built-in tool called 'git-svn'. Its main purpose is to migrate from Subversion to Git, or at the most do one-way syncing. In my case, two-way synchronization was the only option, since the Subversion repository was also used by another independent team.

After some experimentation I created a script that was able to do 2-way synchronization without too many problems, though this was just on my local machine, and we needed a solution for the entire team.

Jenkins, Containers & Gitlab to the rescue!
What I came up with was to build a Jenkins pipeline script that does the heavy lifting of synchronizing between git and subversion, while also integrating a Continuous Integration pipeline so that the main codebase is kept stable and thus *should* never break.

Fortunately the client was nice enough to provide us with a bare-bones RHEL 7 VM, and from there, I created a containerized CI/CD suite with containers for Jenkins, Gitlab, Sonar and a local Artifactory for snapshot build management. I created 2 Jenkins pipeline scripts for each project, one to do the git-svn synchronizations, and one to run a ci/cd pipeline that sets up an ad-hoc Docker container for the app (and one for the DB), and runs build & tests against them.

This CI/CD pipeline is triggered both by opened merge requests into the develop branch (no one is allowed to commit directly to develop/master branches), and by the git-svn sync pipeline script. The latter synchronizes between the master branch of git and the subversion repository and triggers the CI/CD pipeline in reverse order, that is, it rebases SVN code into master, and after the CI/CD pipeline successfully passes, merges the code into develop. This way, the develop branch is protected from broken build commits both from SVN and from feature branches.

I created a demo project with examples of the pipeline script and a full-fledged containerized solution you can use as a starting point:
https://github.com/kdubois/git-svn-pipeline

'Add More' form fields with javascript / jquery

Posted on by
7

A few years ago I wrote a post on how to add more fields without reloading the page using javascript, but that method was kind of clunky and involved a hack to get it to work with the DOM.  The following is a much easier way, and (as far as I've tested) compatible with all major browsers.

First off, download the latest version of jquery and include it on your page, or just include the following line if you trust google to host your jquery file:

<script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js"></script>

Here's a sample html form – adjust according to your needs:

<form action='mypage' method='post' accept-charset='utf-8'>    
 <ul id='myfields'>
  <li>
   <input type='text' name='mytextfield[]' /> 
  </li>
 </ul>
 <div id='addmorelink'>
  <a href='javascript:addField()'>add more</a>   
 </div>
</form>


What's important is the ul with the 'myfields' id, and the link that calls the 'addField()' js function.


The javascript code is very short and straightforward – basically copy your list item with the form field code into the content of the 'newContent' variable, make sure you escape quotes if you enclose the string with the same quotes as in your form field, and then use the jquery 'append' function to add the next field to your form, et voila, that's it!


<script type="text/javascript"> 
 function addField(){
  var newContent = "<li><input type='text' name='mytextfield[]'/></li>";
  $("#myfields").append(newContent); 
 }
</script>


I haven't mentioned one last piece that's going to come in handy on the server side where you're going to process the fields.  You probably noticed the [] brackets behind the field name in the form – that turns the field into an array.  On the server side, depending on what programming language you use, you just need to handle your post variable as an array – so to retrieve each value you'll need to loop through the array.  Assuming you're programming in php and you're not using a framework that handles post variables differently:


<?php
 if(isset($_POST['mytextfield'] &amp;&amp;  is_array($_POST['mytextfield']){
  foreach($_POST['mytextfield'] as $value){ 
   // do something with each $value 
  }
 } 
?>