Simplify Spring Boot Access to Kubernetes Secrets Using Environment Variables

This blog post is a follow-up to a previous blog post titled “Simplify Spring Boot Access to Secrets Using Spring Cloud Kubernetes“. Despite the downsides I mentioned, I already hinted at a more straightforward solution that utilizes environment variables. The plan is to get everything into the Pod with as little configuration effort as possible.

So, I promised a twist, and here it is, thanks to one of my colleagues who pushed me in this direction. Kubernetes gives you yet another tool to handle Secrets in environment variables. This time, it is more convenient since you only point it to the complete Secret, not just a single value. Kubernetes will then make all key-value pairs available as individual environment variables.

Read More »

Simplify Spring Boot Access to Secrets Using Spring Cloud Kubernetes

This topic has its origin in how we manage Kubernetes Secrets at my workplace. We use Helm for deployments, and we must support several environments with their connection strings, passwords, and other settings. As a result, some things are a bit more complicated, and one of them is the access to Kubernetes Secrets from a Spring Boot application running in a Pod.

This blog post covers the following:

  1. How do you generally get Secrets into a Pod?
  2. How do we currently do it using Helm?
  3. How can it be improved with less configuration?
  4. Any gotchas? Of course, it is software.

I will explain a lot of rationales, so expect a substantial amount of prose between the (code) snippets.

Read More »

Apache Commons CLI Handling of –help

An odd thing about Commons CLI is that it has no built-in concept of a “–help” option. Other libraries, like JCommander do (which had other problems, or I would not have bothered with Commons CLI). As a result, you have to build it on your own. It is not enough to include it with all the other application options, especially if you use required arguments. Then it is impossible to only set the Help option.

You must implement a two-step process. See this demo application on GitHub that I created for another blog post. It shows this in action.

First, only parse for the Help option, and if it is present, print the help text and exit the application. To print the complete help text, you must add the other parameters first, though. Otherwise there would be only “–help”.

final var applicationOptions = example_2_Options();

final var options = example_2_Help();
final var cli = parser.parse(options, args, true);

if (cli.hasOption(help)) {
    // Append the actual options for printing to the command-line.
    applicationOptions.getOptions().forEach(options::addOption);
    new HelpFormatter().printHelp("external-config-commons-cli", options);
    return;
}

Second, if no help is requested, parse for the application options.

final var cli = parser.parse(applicationOptions, args, true);

applicationOptions.getOptions().forEach(opt -> {
    if (cli.hasOption(opt)) {
        System.out.printf("Found option %s with value %s%n",
                opt.getOpt(), cli.getOptionValue(opt));
    }
});

Thank you very much for reading. I hope this was helpful.

Spring Boot Externalized Config on Command Line With Apache Commons CLI – Missing Required Option

I know this title is a bit of a mouthful, but you need to get all the keywords in for Google to do its magic 😉. In the previous blog post, I mentioned that I would take another look at this topic through the lens of a programmer that uses Apache Commons CLI for command-line argument handling. In a project for work, I noticed some odd error messages claiming that a command-line option did not have a value assigned to it, although it obviously did.

A more extensive set of examples can be found in the README file on GitHub, together with the code.

The sole reason for this blog post is how unknown parameters from the view of Commons CLI can mess up the parsing. The demo application defines two required Options – one for input (“-i” or “–input”) and one for output (“-o” or “–output”). Consider this command where I also set a Spring configuration setting.

% java -jar target/external-config-commons-cli-1.0.0.jar --spring.config.additional-location=src/config/application-mac.yml -i in -o out
-> AppRunner.run() Command Line Arguments
Argument: --spring.config.additional-location=src/config/application-mac.yml
Argument: -i
Argument: in
Argument: -o
Argument: out
-> ExternalConfigProperties
Input path: /Users/mac/thecode
Output path: /Users/mac/slinger
-> Parsing Help With Apache Commons CLI
-> Parsing Arguments With Apache Commons CLI
Missing required options: i, o

Both options are clearly there. The raw output of the String… args array shows that. By default, Commons CLI complains about unknown options. I disabled that behavior by setting stopAtNonOption to true. The parameter’s name makes no sense to me because it does not stop, but I might misinterpret something.

Either way, I assume that Commons CLI expects an option and a value by default. –spring.config.additional-location=src/config/application-mac.yml is a continuous string, an option without a value – at least to Commons CLI. Then it reads -i as the value to that option, and from there, the parsing goes south. The actual options are interpreted as values now.

Note, though, that Spring still accepts the configuration setting.

How can we fix that? There are two ways to do that:

  1. Add the Spring arguments at the end of the command line.
  2. Use the JVM-style Spring arguments with “-D”, as alluded to in the other blog post.

Putting the argument at the end:

% java -jar target/external-config-commons-cli-1.0.0.jar -i in -o out --spring.config.additional-location=src/config/application-mac.yml
-> AppRunner.run() Command Line Arguments
Argument: -i
Argument: in
Argument: -o
Argument: out
Argument: --spring.config.additional-location=src/config/application-mac.yml
-> ExternalConfigProperties
Input path: /Users/mac/thecode
Output path: /Users/mac/slinger
-> Parsing Help With Apache Commons CLI
-> Parsing Arguments With Apache Commons CLI
Found option i with value in
Found option o with value out

Using the JVM-style:

% java -Dspring.config.additional-location=src/config/application-mac.yml -jar target/external-config-commons-cli-1.0.0.jar -i in -o out
-> AppRunner.run() Command Line Arguments
Argument: -i
Argument: in
Argument: -o
Argument: out
-> ExternalConfigProperties
Input path: /Users/mac/thecode
Output path: /Users/mac/slinger
-> Parsing Help With Apache Commons CLI
-> Parsing Arguments With Apache Commons CLI
Found option i with value in
Found option o with value out

Thank you very much for reading. I hope this was helpful.

Spring Boot Externalized Config on Command Line

Spring Boot applications do not always have to serve as a web service located on the Internet. You can also use Spring Boot (or Spring without the Boot) for a command-line utility. I was recently faced with this task, and one requirement for the tool was to support setting a profile-specific configuration on the command line. This isn’t earth-shattering per se since that is a regular Spring feature. The goal was to provide a profile-specific configuration file on the command line that is not bundled in the application.

Imagine developing a Cloud service and running different environments for the different phases of your project – one for development tests, a staging environment, and, finally, the production environment. Connecting to the different environments may require secrets you do not want to be bundled in the application – and, thus, the source tree.

Now, you could roll your own configuration file reader. But wouldn’t it be nice to make full support of Spring’s @Value annotation or @ConfigurationProperties classes?

Read More »

Create Native Java Executable using jpackage – Sort of

I have always been the kind of developer who prefers to use native code and write native code. My background is in C++, and I have worked with Microsoft’s WinAPI early in my career. That is to say: I like it fast, and I do not mind going to lower levels.

I am not stuck in the past, though, and as such, I, too, have evolved with the times. I still like C++, but I also see how languages like Java and its great tooling can boost productivity in comparison. As a result, I write code fast. Java is the tool of the trade at my current job, and performance usually is not a problem anymore. The JVM has improved, and computer hardware has, so performance is usually not an issue anymore.

There is one little problem, however: Usage. Let me explain.

Read More »

Apple Silicon M1 for Software Development: Java, C++ with Qt

Apple’s laptops have been making quite the splash since the end of 2020 and have made a massive comeback as a professional tool one year later with the M1 Pro and Max designs. Most of the reviews I have seen focus on the editing and rendering capabilities of these new MacBooks. A few reviewers throw a compile test in the mix, but compiling Chromium or any other huge project is only a part of the equation. Developers don’t just compile code; they also use tools and IDEs to develop their software.

Being new to the M1 world, I wanted to recap my experiences so far briefly. I use Java professionally, and I also have a C++ application based on the Qt framework that I wrote an eon ago and still use productively. Being a former C++ professional, I am about native performance, and I like native software. Therefore, I intended to utilize as many Apple Silicon-native tools as possible. Luckily, one year after its release to the desktop world, the most popular applications have caught up. Let me go through my tool suite one by one.

Read More »

C# Delegate, Action, Func, Predicate Explained

Depending on your entry point to delegates, the documentation might look a tad confusing at first. For me, it was The delegate type section of the C# language reference. It throws around terms like Action, Func, Events, custom delegate types. Predicate is also related to this topic, and this was what I was looking for.

Let me briefly explain what all those words mean and how they relate to delegate. Then I will explain why I was looking into this.

Short teaser: “Named Predicate”, like a Hibernate Named Query.

Read More »

Emulate Java Enums With Values in C# (Pt. 2, Improved With Conversion Operator Overload)

In a previous blog post, I demonstrated how Java’ enums that contain one or more values/objects can be emulated with C#. One thing bothered me, though: the switch statement and how inconvenient it was to determine the proper type. Worst of all, it was not type-safe. In my simple example, it was easy because I was using strings. Imagine your fake-enum does not contain a string to quickly identify the instance.

Well, there is a prettier workaround – and it involves an actual enum. I was thinking about how the same could be done in C++ and in C++, you can have type conversion operators. Then I searched if such a feature also exists in C#, and sure enough, it does.

Read More »

Emulate Java Enums With Values in C#

Update August 28, 2021

I have written a follow-up that improves on the following solution using a type conversion operator overload.

When I started dabbling in C#, I wondered if it supports values in enums. In Java, an enum instance can have properties (called fields in Java lingo) associated with the enum’s literals. By taking advantage of this feature, you can encode more information in an enum, like a string, for example, or a constant number. You can even embed instantiated class objects, maybe to associate an object factory with a literal.

In my use case, I wanted to achieve a form of a key-value-pair mapping. I require certain illegal characters in the NTFS file or directory names to be replaced with a given code. I use HTML encoding for my needs because I can simply look up the values online if I need to.

Here is the Java reference example. First, let me start with the basic enum definition (I use Lombok to auto-generate boilerplate code like the constructor and accessors).

@Getter
@RequiredArgsConstructor
enum CharacterReplacementCode  {
    COLON(":", "&58;"),
    POUND("#", "&35;"),
    QUESTION_MARK("?", "&63;"); 

    private final String character;
    private final String replacement;
    
    @Override
    public String toString() {
        return String.format("Character '%s' substituted by code '%s'", character, replacement); 
    }
}
Read More »

Terraform Azure Error SoftDeletedVaultDoesNotExist

I just ran into a frustrating error that seemed unexplainable to me. My goal was to replace an existing Azure Resource Group with a new one managed entirely with Terraform. Besides a few other errors, this SoftDeletedVaultDoesNotExist was incredibly confusing because no more Key Vaults were found in the Resource Group’s list of resources.

Error: creating Vault: (Name "my-fancy-key-vault" / Resource Group "The-Codeslinger"): 
keyvault.VaultsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- 
Original Error: Code="SoftDeletedVaultDoesNotExist" 
Message="A soft deleted vault with the given name does not exist. 
Ensure that the name for the vault that is being attempted to recover is in a recoverable state. 
For more information on soft delete please follow this link https://go.microsoft.com/fwlink/?linkid=2149745"

with module.base.azurerm_key_vault.keyvault,
on terraform\key_vault.tf line 9, in resource "azurerm_key_vault" "keyvault":
    9: resource "azurerm_key_vault" "keyvault" {

That is because it was soft-delete enabled. And it was the Key Vault from the other Resource Group that I previously cleared of all resources, not the new Resource Group.

Using the az CLI you can display it, though.

> az keyvault list-deleted
[
    {
        "id": "/subscriptions/<subscription-id>/providers/Microsoft.KeyVault/locations/westeurope/deletedVaults/my-fancy-key-vault",
        "name": "my-fancy-key-vault",
        "properties": {
            "deletionDate": "2021-08-02T09:39:29+00:00",
            "location": "westeurope",
            "purgeProtectionEnabled": null,
            "scheduledPurgeDate": "2021-10-31T09:39:29+00:00",
            "tags": {
                "customer": "The-Codeslinger",
                "source": "Terraform"
            },
            "vaultId": "/subscriptions/<subscription-id>/resourceGroups/My-Other-ResourceGroup/providers/Microsoft.KeyVault/vaults/my-fancy-key-vault"
        },
        "type": "Microsoft.KeyVault/deletedVaults"
    }
]

And finally delete it.

> az keyvault purge --name my-fancy-key-vault

After that, it is gone.

$ az keyvault list-deleted
[]

Another option seems to be the Azure Portal, but I discovered this only after removing it on the command line.

Qt6 QtCreator Crash After Install on Ubuntu 21.04

Hopping Linux distributions, I came to Ubuntu 21.04, and one of the first things I do is install Qt manually. I have described the process in a previous blog post on Linux Mint, and it is the same for Ubuntu. Except for a tiny detail. On Ubuntu, the bundled QtCreator immediately crashes and triggers a "Send Diagnostic" dialog.

$ /opt/Qt/Tools/QtCreator/bin/qtcreator
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was 
found. This application failed to start because no Qt platform plugin could be 
initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, 
xcb.

The fix is simple.

sudo apt install libxcb-xinerama0

I hope this helps. Thank you for reading.

Sign and Verify JWT With Hashicorp Vault REST API

Cryptography is complicated in more than just one way. Therefore, it is commonly recommended not to roll your own, but instead, employ tried and tested methods. Unless you are an experienced cryptographer, it is likely to overlook crucial things, for example, when to authenticate an encrypted message – before decrypting or after? This blog post is about JSON Web Tokens that are digitally signed with an RSA key. Instead of implementing the signing and verification code yourself, you should be using a dedicated server component to do the complex crypto for you, like Hashicorp Vault.

Read More »

Write ID3 mp3 Tags, Cover Art with Python and eyed3

In March 2019, I wrote about using Python with the "pytaglib" library to read and modify audio files’ metadata. This solution worked nicely for what I needed at the time, but as I rewrote my WAV-to-MP3 conversion in Python, I found that it lacked support for adding cover art to the files. After a bit of research, I found eyed3. It also comes with a nice side-effect: it does not require installation of the Taglib C++ library.

Installation is simple.

pip install eyeD3

Additionally, on Windows, you need this, too:

pip install python-magic-bin

For more details, visit the installation guide. Like my old pytaglib post, I will keep this one short and only show code samples for the most relevant tasks.

Before you do anything, import the eyed3 library, of course.

import eyed3

Load a file.

song = eyed3.load(file)

If the file does not yet have any tags, you must create them before setting any metadata.

if not song.tag:
    song.initTag()

Note that this erases existing tags, so only call that if you start from scratch. Now you are ready to set the individual tags or read them if you need them.

song.tag.artist = "Behemoth"
song.tag.album = "The Satanist"
song.tag.genre = "Black Metal"
song.tag.recording_date = 2014
song.tag.track_num = 4
song.tag.title = "Ora pro nobis Lucifer"

The important piece for me, write cover art.

with open(cover_art_filename, "rb") as cover_art:
    song.tag.images.set(3, cover_art.read(), "image/jpeg")

The value 3 indicates that the front cover shall be set. See the documentation for other values. If you are an iTunes user, then select 0 for "Other". iTunes does not seem to like "Front Cover" 🙄.

You may run into a warning message like the following if you use genre names not defined in the ID3 specification.

eyed3.id3:WARNING: Non standard genre name

You do not need to worry about that. If it annoys you, add the following line to your code. eyed3 still writes the tag without any issue.

eyed3.log.setLevel("ERROR")

Convert Java POJO With Protobuf field to JSON Using Jackson

In this blog post, I will explain how to convert a regular Java class that contains a Protobuf message field to JSON using the Jackson library, for example, in a Spring Boot application as a return value of an @Controller method.

You might wonder, how this is such a big deal? After all, you can create complex POJO hierarchies, and Jackson will pick them up just fine. Well, maybe this error message will convince you.

o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [
    Request processing failed; nested exception is org.springframework.http.converter.HttpMessageConversionException: 
    Type definition error: [simple type, class com.google.protobuf.UnknownFieldSet]; 
    nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle 
    (through reference chain: com.thecodeslinger.ppjs.web.dto.AwesomeDto["awesomePowerUp"]
        ->com.thecodeslinger.ppjs.proto.AwesomePowerUpOuterClass$AwesomePowerUp["unknownFields"]
        ->com.google.protobuf.UnknownFieldSet["defaultInstanceForType"])] with root cause

Java classes that you create with the Protobuf compiler require their JSON converter JsonFormat.Printer. So, how can we get Jackson and JsonFormat.Printer love each other and have a wedding together?

Simple: we create a custom JsonSerializer.

public class ProtobufSerializer extends JsonSerializer<Message> {

    private final JsonFormat.Printer protobufJsonPrinter = JsonFormat.printer();

    @Override
    public void serialize(Message anyProtobufMessage, JsonGenerator jsonGenerator, SerializerProvider serializerProvider)
            throws IOException {
        // The magic sauce: use the Protobuf JSON converter to write a raw JSON
        // string of the Protobuf message instance.
        jsonGenerator.writeRawValue(protobufJsonPrinter.print(anyProtobufMessage));
    }
}

This class is very simple and can work with any Protobuf Message class instance. This way, it is universal and only needs to be written once. The main ingredient is the jsonGenerator.writeRawValue method that takes the input without modification. Since we already ensure a proper JSON format using Protobuf’s converter, this is no problem in this case. Otherwise, be careful with this method.

The last step is to annotate the Message field in the POJO, so Jackson knows what to do.

@JsonSerialize(using = ProtobufSerializer.class)
private final AwesomePowerUpOuterClass.AwesomePowerUp awesomePowerUp;

You can find a complete working example that uses Spring Boot and a REST endpoint on my Github account.