How adding a functional utility helps to avoid code duplications and leads to more readable code.

In Spring Data Redis, we support multiple Redis clients – Jedis and Lettuce.
Commands can be invoked either directly, in a transaction, or using pipelining.

Direct commands get invoked – as the name probably reveals – directly by calling a client method and returning the result. In some cases, results require post-processing because the Redis driver reports a Long while Spring Data Redis would like to return Boolean or the driver returns a driver-specific type that requires conversion in a Spring Data type.

In my previous blog post I started an experiment with using Project Loom. The post outlined the first steps to make use of virtual Threads on a best-effort basis (i.e., without rewriting the entire libraries involved, instead fixing issue by issue until it works™).

(more…)

This article walks you through a experiment that uses a Spring Boot application with Virtual Threads. Having access to early access builds is the perfect opportunity to take a look what it takes to use virtual threads as worker threads. With all customizations in place, we issue a few request to verify that our application is running. Finally, we put a bit of load onto the application to see how memory consumption and the number of kernel threads develop over time.

Follow-up post: Carrier Kernel Thread Pinning of Virtual Threads (Project Loom).

(more…)

Spring Framework announced recently to ship with support for reactive transaction management.
Let’s take an in-depth look at how this works for R2DBC, the reactive specification for SQL database access.

(more…)

R2DBC Tour took place in Mid- and North-parts of Germany. R2DBC standardizes reactive integrations with SQL databases as Open Source initiative.

Reactive Relational Database Connectivity (R2DBC) started out as idea to see what’s possible when thinking about reactive programming and SQL databases. Since the last months, R2DBC grew beyond that idea. It inspired driver maintainers and client library authors as common language: A specification for drivers and an interface for library authors so they can build client libraries on top of a vendor-independent SPI. Client libraries are tools that we as application developers need to keep productivity high and not to solve problems that are solved already.

(more…)

Last year I announced the R2DBC Java User Group and recently I started the tour visiting various user groups and conferences. Reactive Relational Database Connectivity lifts a roadblock when integrating with SQL databases. It’s an Open Source standardized API across multiple database drivers and an ongoing specification effort hosted on GitHub.

(more…)

Imperative code eats threads at the pace of incoming requests while Software is eating the world. This post discusses the assumptions for reactive programming on the JVM and what this means for integrations – in particular, relational databases.

The motivation to come up with a post is the constant increase in reactive programming adoption while some major building blocks are not yet available – in particular, the question: What about relational databases?

(more…)

R2DBC is an endeavor to bring a reactive programming API to relational data stores and stands for Reactive Relational Database Connectivity. It was first announced at SpringOne Platform 2018. I’m happy to announce my R2DBC tour across Java User Groups in Germany, Switzerland, and Austria. I will explain what R2DBC is, how it was founded and what to expect from a reactive relational database integration. We will speak about the SPI, various drivers, and about Spring.

(more…)

Redis Streams have made it into Redis’ unstable branch since the beginning of this year, and the first clients start with the adoption of Redis Streams APIs. That makes it an excellent time to take a look what Redis Streams provide and how you can use them from a client perspective.

(more…)

After a 13 months development phase and 208 solved tickets, it is my pleasure to announce general availability of Lettuce 5.0. This is a major release coming with several breaking changes and new interesting features and Java 9 compatibility.

Get the release from Maven Central

<dependency>
    <groupId>io.lettuce</groupId>
    <artifactId>lettuce-core</artifactId>
    <version>5.0.0.RELEASE</version>
</dependency>

or download the release bundle from GitHub.

(more…)

I released today logstash-gelf 1.11.1. This release fixes a bunch of bugs and brings minor enhancements.

(more…)

Heckenlights is an interactive Christmas lights installation that can be controlled online by just visiting heckenlights.org. Heckenlights consists of over 3000 LED lights that are controlled via MIDI in 12 tracks. Each MIDI note switches the lights on and off.

It all started in 2012 with the idea to control a Relay with MIDI and lead to what it is today. Now, this season is the last one for Heckenlights. It’s been a great fun building Heckenlights and even more stepping outside, opening heckenlights.org on a smartphone and triggering the lights to play.

I’m moving next year to a new house. Heckenlights (originating from Heckenpfad, the address I live) is no more appropriate then.

You can see and operate Heckenlights until Jan 6th, 2017 between 5 pm and 10 pm (CET). After that date, it will be shut down.

Resources:

• midi-relay at Github
• Heckenlights at Github

Redis is a data store supporting over 190 documented commands and over 450 command permutations. The community supports actively Redis development; each major Redis release comes with new commands. This year Redis was opened up for 3rd party vendors to develop modules that extend Redis functionality. Command growth and keeping track with upcoming modules are challenging for client developers and Redis users.

(more…)

I just released lettuce 4.2.2.Final.

This is a bugfix release that fixes several issues. Updating is recommended for
setups with frequent reconnects or frequent cluster topology changes during runtime.

(more…)

This post explains the use of a pooled GZIP OutputStream. The initial
motivation were performance measurements of the GZIP’ing inside of logstash-gelf.

(more…)

Passwords, API keys and confidential data fall into the category of secrets. Storing secrets the secure way is a challenge with limiting access and a true secure storage. Let’s take a look at Hashicorp Vault and how you can use it to store and access secrets.

(more…)

Blue/Green deployments are a method of deploying with zero downtime. This is possible on Pivotal’s CloudFoundry using either the CLI, Maven, or Gradle. Matt Stine wrote earlier on this topic how to do blue/green deployments using the CLI.

(more…)

Redis transactions allow to group multiple commands and to execute them sequentially. The whole transactional state is isolated from other users and becomes visible once a transaction is committed.

A Redis transaction is different from transactions in, let’s say, relational databases. A Redis transaction feels more like a queue/stack of commands because commands are queued and the execution is deferred. This is true for reads and writes. The real surprise is the read commands because any return value is returned only upon transaction execution.

(more…)

I consider myself very lucky for contributing to the open source space. Open source is one of the main building blocks of our digital world. For me, contributing is sharing, and sharing is giving back. I started contributing in early 2011. Transparency, quality, and purpose are my main reasons for my open source involvements. That is how I got in touch with the Spring Data team.

(more…)

I just released lettuce 3.4.Final and lettuce 4.1.Final. You can find the full details about 3.4.Final and 4.1.Final on Github.

These release contain numerous features and bugfixes. Lettuce introduces reusable client-resources, an EventBus, client metrics, and support for newly introduced commands. This versions work with Redis 3.2 RC3 but Redis expects a change in the format of CLUSTER NODES. So watch out for a new release of lettuce as soon as Redis 3.2 RC4 or a final is released.

(more…)

I just released lettuce 3.3.1.Final and lettuce 4.0.Final. I’m going to tell you in this post more about 4.0. You can find the full details about 3.3.1.Final and 4.0.Final on Github.

lettuce 4.0 is a major release that introduces numerous changes like stateful connections, the reactive API and many more. Lettuce 4.0 includes all features from lettuce 3.3.

Highlights of lettuce 4.0.Final

• Reactive API
• Stateful connections
• Cross-slot command execution
• Node Selection API/Execution of commands on multiple cluster nodes
• ReadFrom Settings/Redis Cluster slave reads
• Custom commands

This release contains some breaking changes. You may want to consult the wiki at Migration from 3.x to 4.x to check the migration guide.

(more…)

I released spinach 0.2 today.

This release addresses minor issues and allows to use lettuce 3.3.Final. It adds support for more Disque commands and eases creation of a DisqueURI.

(more…)

lettuce 4.0.Beta2 features ReadFrom Settings. ReadFrom affects how lettuce routes read operations to the members of a Redis Cluster.

By default, lettuce routes its read operations to the master node. Reading from the master returns the most recent version of the data because write operations are issued to the single master node. Reading from masters guarantees strong consistency.

You can reduce latency or improve read throughput by distributing reads to slave members of the Redis cluster for applications that do not require fully up-to-date data. ReadFrom is set per connection, meaning you can use different connections with different ReadFrom settings.

(more…)

I’m very excited to present you lettuce 3.3.Final. Read on for the details or jump to the end to find the summary and the download links.

(more…)

Implementing clean architecture is amazing. A clean architecture fundamentally changes the way you create software and perform changes to existing code. The architectural style, also known as Jacobson architecture or Onion architecture, is based on the business rules. Meaning the application model and use cases. These parts become the explicit core of such an application. The core is agnostic against the database, the web, and frameworks, which are then just plugins to the application.

(more…)

The Geo-API of the upcoming Redis 3.2 release allows to maintain a set (backed by a Redis sorted set) of Geo points described by WGS84 coordinates. You can add and query set members using the new Geo-API. Use ZREM to remove members from the
until https://github.com/antirez/redis/issues/2674 is resolved.

(more…)

I released today two betas of lettuce: 3.3.Beta1 and 4.0.Beta1. What is the difference between those versions? Both use netty and a very similar core to serve as Redis driver. The 4.0 branch introduces a Reactive API, stateful connections, improved API and command execution on multiple cluster nodes. 4.0 also introduces a set of breaking changes. The 3.x branch is supported for at least a year, new Redis commands will be supported in both branches. 4.0 is the next generation of lettuce.

(more…)

CDI is one of the best additions to Java EE of the recent. This opinion is widely shared amongst users and integrators. CDI 1.2, the current version, was released in April, 2014. Now in mid-2015, we face the early draft review of the CDI 2.0 spec. CDI 2.0 will run on Java 8 and higher.

(more…)

I released the first version of spinach, a scalable Java Disque client based on netty. spinach is built on top of the lettuce client which provides SSL and Unix domain connections in addition to plaintext connections.

Basic usage

DisqueClient client = new DisqueClient("host");
DisqueConnection<String, String> connection = client.connect().sync();
DisqueCommands<String, String> sync = connection.sync();
String jobId = sync.addjob("queue", "body", 1, TimeUnit.MINUTES);

Job<String, String> job = sync.getjob("queue");
connection.ackjob(job.getId());

Features

• Works with Java 6, 7 and 8
• synchronous and asynchronous APIs
• SSL and Unix Domain Socket connections
• Codecs (for UTF8/bit/JSON etc. representation of your data)

For complete information on spinach see the websites:

• http://github.com/mp911de/spinach
• http://spinach.paluch.biz.

Links

• Release on Github
• Artifact overview on Maven Central
• Download spinach 0.1
• Wiki
• Maven Site (Javadoc)

Maven coordinates:

<dependency>
    <groupId>biz.paluch.redis</groupId>
    <artifactId>spinach</artifactId>
    <version>0.1</version>
</dependency>

Any feedback is appreciated or create issues on GitHub.

In May 2014, I needed a Redis client. I had to collect log events within Redis (see https://github.com/mp911de/logcapture) during integration tests. The nature of integration tests and log events is, when tests are running, a lot is logged, but after the tests the connection is idle. Several clients caused connections to run into defective states, and I did not want remote log capturing kills my integration tests. So I took the best client amongst them all and started to fix the issues that were show stoppers.

Now, about a year later, lettuce 3.2 is the most recent version, three versions were released since then. Developing lettuce is amazing. It’s by far the most advanced low-level client. There are others like Spring Data Redis or Redisson that provide high-level functionality. I’m not talking about those high-level clients. I guess, my biggest mistake was connection pooling. I was supercilious, and I was convinced, that connection pooling is a killer feature and a must. I had only my use case on my mind. @wg did not like connection pooling, too, and most probably it is the reason, why he never merged my pull request.

However, my best decision was to start working on lettuce. I learned a lot, and I was able to support nearly all Redis 3.0 features: Sentinel, Cluster and a ton of commands. The features are well received, and the downloads increase every month.

The question might arise: Where is the journey heading? What are the strategy and the roadmap?

I want lettuce to be the most easy-to-use, advanced, and, flexible Java Redis client. I want to provide a synchronous, an asynchronous and a reactive API. Spinach, the Java Disque client, is built on top of lettuce. A pull request for Spring Data Redis, upgrade lettuce to 3.2, is already in progress.

The roadmap for lettuce 4.0:

• Advanced Cluster API: Provide means to run commands on a selected set of cluster nodes
• Pipelining for specific cluster commands
• Improve RedisFuture
• Add a reactive API
• Drop Java 6 and 7 support and maintain the 3.x branch for a certain time.
• Streamline connection resources, so that users can use different API’s for the same resource
• More community feedback

I learned that plain Java futures are not flexible enough, that’s the reason, why RedisFuture is a ListenableFuture. Lettuce 4.0 will use CompleteableFutures under the hood, and this will change the behavior. Currently RedisFuture.get does not throw an exception when the command errored within Redis. The ´RedisFuture.getError´ method provides the error text. CompleteableFuture‘s require to complete either with a result or with an exception, and that, will be a breaking change for async users.

Although downloads and the user base grow, the community around lettuce is far too small. I am grateful for every feedback and discussion on lettuce and it’s API.

I want the community to grow. I created recently the mailing list/Google Group lettuce-redis-client-users and updated the contribution readme. Github Issues and the mailing list are great starting points to contribute.

This release features Unix domain sockets on linux-x86_64 systems. Local connections to a Redis instance are now possible without the use of the network. A second feature with a huge impact is client options. Client options allow to control behavior on a fine-grained base. It is now possible to turn off auto-reconnect, validate the connection with a PING before activating the connection for client usage and many more.

(more…)

This release features Unix domain sockets on linux-x86_64 systems. Local connections to a Redis instance are now possible without the use of the network. A second feature with a huge impact is client options. Client options allow to control behavior on a fine-grained base. It is now possible to turn off auto-reconnect, validate the connection with a PING before activating the connection for client usage and many more.

(more…)

It has been over a year ago the project started that I’m currently working on. This project had the goal to create an archiving service for grocery point of sales receipts. We ended up with creating an outstanding system that might become the new blueprint for software applications within the customers’ software development departments.

I’m now on a Java/JBoss/AngularJS project, that is heading towards it end. A triumphal end, since we created an outstanding software system in many various ways. The most impressive, the application has the potential to be the blueprint for future applications. It will influence how applications will be developed and operated in this particular customers’ software development teams. Here is the story.

TL;DR;

(more…)

You might have noticed it if you are a developer. In rare occasions, there might be the case a particular artifact runs on your machine. But not anywhere else. This issue is known as the “It works on my machine” problem. As soon as this happens, …

There are several causes, why this might happen. Either a different build process is used, the build environment is different, the way how you deploy is slightly different, the runtime environment differs from the others, the config looks like different, … or a thousand more reasons might have caused that the artifact runs just on one machine.

(more…)

This release introduces support for SSL, enables lambda expressions on streaming channels and a lot of bugfixes and resilience changes.
lettuce extensibility is improved. You can override certain parts of the client to supply classes (connection classes) that behave the way you need by extending RedisClient or RedisClusterClient.
You can reset the internal connection state in case you run into broken connections (invalid connection state due to SSL tunneling or others) by calling the reset() method on your connection.

(more…)

Couchbase Essentials is a book written by former Couchbase Inc. developer John Zablocki. The book claims to be for developers, but it is valuable to software architects as well. The book starts with a lot of marketing about Couchbase, explaining the history from CouchDB and memcached. This chapter will guide its readers through a very detailed installation procedure that reminds on a very detailed Couchbase manual. Nearly every click is explained.

(more…)

Do you want to test your REST API as soon as possible and without any hassle? The earliest point in the code where tests can occur is within the commit stage, where you can run unit tests. I want to present you an approach how to test REST API as soon as possible in an easy way. The example code is based on JUnit and RESTEasy. It does not require any external database or heavyweight application servers which means you can run it isolated and out of the box.

(more…)

One day a coworker came to me convincing me: Let’s do something cool with internet of things and sensors. Until now, I did not know anything about IoT sensors but my first idea was a sonic sensor that tells the distance.

(more…)

I got in touch with Couchbase a year ago. I attended a presentation of the Couchbase guys and learned that Memcached became a part of Couchbase. Until now I never had a chance to use Couchbase for more than just a play around. Recently I stumbled upon a new Book about Couchbase: Couchbase Essentials written by John Zablocki. I will let you know what I think about it and post my review here. Stay tuned.

I released lettuce 3.0.3. This is a bugfix release fixing a memory leak in pubsub and adds stability when reconnecting.

(more…)

Debugging tests is always tricky. Debugging Selenium Tests executed with JBehave is harder. The JBehave tests are executed and stack traces are written to logs. In many cases, you can’t reproduce the case or you need to fiddle around, where to set your breakpoint. This is not effective. This is annoying. I want to present you a slightly different and improved approach for debugging Selenium/Selenide tests. You will need either CDI, a Spring Context or Google Guice as prerequisite.

(more…)

I released lettuce 3.0.2. This is a bugfix release fixing a memory leak in pubsub.

This release contains:

Fixes

• Memory leak #21, #22

Other

• Streamline build and release

Links

• Release on Github
• Artifact overview on Maven Central
• Download lettuce 3.0.2.Final
• Wiki
• Maven Site (Javadoc)

Maven coordinates:

<dependency>
    <groupId>biz.paluch.redis</groupId>
    <artifactId>lettuce</artifactId>
    <version>3.0.2.Final</version>
</dependency>

<dependency>
    <groupId>biz.paluch.redis</groupId>
    <artifactId>lettuce</artifactId>
    <version>3.0.2.Final</version>
    <classifier>shaded</classifier>
</dependency>

Any feedback is appreciated or create issues on GitHub.

I released today logstash-gelf 1.6.0. This release contains support for GELF 1.1 spec, Redis Sentinel and the Datenpumpe.

(more…)

Some years ago I moved into my own home. At that time I had the idea to setup christmas lights in our garden. Maybe not as large as Carson Williams setup, but at least something. And well, you know, it grew from year to year. Now I came to the point where I want give you control over the lights.

(more…)

I released today logstash-gelf 1.5.4. This release carries some internal improvements if you plan to extend log appenders and GELF senders.

(more…)

Elasticsearch, Logstash and Kibana are widely used for log management. I wrote in earlier posts about submitting application logs towards logstash. Let’s approach access logs today.

(more…)

A choice of decision for a NoSQL data store depends on several factors. It’s no longer a hot-or-not decision. The big hype is over, reality comes back now.

(more…)

lettuce 3.0.1 was released. This is a bugfix release fixing some major bugs.

This release contains:

Fixes

• Test improvements for more stability
• URI for sentinel connections
• Provide static syncHandler
• Handle premature channelInactive while ongoing close (avoid NPE)
• Fix signature of clusterSlaves #18
• Fix SCAN command with scan args

Enhancements

• Optimized CRC16 calculation (thanks to Jedis for co-working)
• Use fine grained exceptions for Timeout, CommandExecution and Connection

Links

• Release on Github
• Artifact overview on Maven Central
• Download lettuce 3.0.1.Final
• Wiki
• Maven Site (Javadoc)

Maven coordinates:

<dependency>
    <groupId>biz.paluch.redis</groupId>
    <artifactId>lettuce</artifactId>
    <version>3.0.1.Final</version>
</dependency>

<dependency>
    <groupId>biz.paluch.redis</groupId>
    <artifactId>lettuce</artifactId>
    <version>3.0.1.Final</version>
    <classifier>shaded</classifier>
</dependency>

Any feedback is appreciated or create issues on GitHub.

lettuce 3.0 was released. Lettuce is a scalable thread-safe Redis client providing both synchronous and asynchronous connections. Multiple threads may share one connection provided they avoid blocking and transactional operations such as BLPOP, and MULTI/EXEC. Multiple connections are efficiently managed by the excellent netty NIO framework. Support for advanced Redis features such as Sentinel, Cluster and Redis data models are included.

Some of the major changes are:

• Support for the latest Redis commands
• Redis Cluster support
• Redis Sentinel support
• Upgrade to netty 4.0.19 (from netty 3.x)
• CDI and Spring support
• Streaming API
• Listenable Futures (using Guava) for true async operations
• Tons of bugfixes, tests and documentation

Lettuce 3.0 is a fork of original lettuce@wg with preserved packages and mostly the same API. The original lettuce is no longer in active development.

What’s next?

My plan is to keep up with tight Redis development and to integrate the new fork into Spring Data Redis. Redis Cluster has to get mature, and the current implementation has to prove in production.

Links

• Release on Github
• Artifact overview on Maven Central
• Download lettuce 3.0.Final
• Wiki
• Maven Site (Javadoc)

Maven coordinates:

<dependency>
    <groupId>biz.paluch.redis</groupId>
    <artifactId>lettuce</artifactId>
    <version>3.0.Final</version>
</dependency>

<dependency>
    <groupId>biz.paluch.redis</groupId>
    <artifactId>lettuce</artifactId>
    <version>3.0.Final</version>
    <classifier>shaded</classifier>
</dependency>

Any feedback is appreciated or create issues on GitHub.

I’ve released today logstash-gelf 1.5.3. This is a bugfix release for Hadoop Hive users. I did not announce release 1.5.2 the last time. Apologies for that.

(more…)

When CDI and Inifispan meet you’ve got the chance to improve your code a lot. Let’s combine both to make all of your CDI beans @Cacheable.

Caching of values usually goes this way:

public String getSomething(String input) {
    String result = cache.contains(input);
    
	if(result == null) {
        result = getValueFromDatabase(input);
        cache.put(input, result);
    }
	
    return result;    
}

This pattern repeats for every value which is cached/retrieved. Methods like the one above contain repetitive conditionals and value retrievals. Using the caching interceptor pattern eliminates the need for repetition. Business methods will be reduced back to their essence and caching becomes an aspect.

@Cacheable
public String getSomething(@CacheKey String input) {
    return getValueFromDatabase(input);
}

(more…)

When releasing lettuce 3.0-Beta3 I was sure this will be the last beta of lettuce 3.0. In the mean time lots of stuff came up: New redis features around clustering, new commands and data structures. On the dev side there was low test coverage and instabilities during the build. This were the reasons to release an third beta. The original plan was to head for final as soon as redis 3.0 is going to be releases.

(more…)

lettuce 3.0.Beta 3 is out now. The new redis commands provide in addition to the basic support also data structures and parsers. This is to support common implementation tasks and to ease adoption.

The Beta 3t is a surprise, because I wanted to release the final after Beta 2. Meanwhile, redis added lots of features and in fact, lettuce was not ready for a final yet. I’ll state more about the details in a further, final announcement post. For now enjoy the Beta 3.

(more…)

Test-driven development is a life-assurance for several software projects.

I’ve started about 10 years ago with test-driven development. Since then I fell into the mantra writing test code then production code run the test and start over. Sometimes even model and interface first then test code then production code and so on. This way less bugs had a chance to remain within production code. But unit testing is not everything. The next level is integration testing, which help to discover even more bugs. As soon as components play together, they start to behave slightly different than assumed in unit tests. Sometimes integration-tests are the only way to test a software (in a sensible way).

(more…)

Logstash, Elastic Search and Kibana gave us power back over our logs. Now it’s time to expand our potential and use well known components to get even more out of the ELK stack.

(more…)

I’ve released today logstash-gelf 1.5.1. This is a bugfix release for JBoss AS7/Wildfly users who use the NDC/MDC.

(more…)

I’ve released today logstash-gelf 1.5.0. This is a major release. logstash-gelf 1.5.0 is the first GELF library which supports the final release of log4j 2.0. This release carries some library updates and internal refactorings to provide also a standalone usage of Gelf message submission.

(more…)

This release is intended to be the last beta release of lettuce. As soon as Redis 3.0 goes final, I’ll release the final of lettuce 3.0.

This release contains:

• Documentation (Javadoc and Wiki)
• Improved test stability for a stable and reproducible build
• Changed redis ports from 63xx to 64xx for testing
• Fixed clientKill because of changed clientList output
• Adjusted resource cleanup to prevent memory leaks
• Prevent refCnt exceptions in case of parallel close & read
• Fixed PubSub API
• Fixed command leaking when client is not yet online/disconnected and commands are issued while the client is in the re-connect phase
• Code cleanup, fixing sonar violations
• Support for SENTINEL MASTERS command
• Support for DEBUG SEGFAULT command
• Support for PUBSUB command
• Support for sorted set commands ZLEXCOUNT, ZRANGEBYLEX, ZREMRANGEBYLEX
• Minor fixes like typos, maven site on http://redis.paluch.biz

Links

• Release on Github
• Artifact overview on Maven Central
• Download lettuce-3.0.Beta2
• Wiki
• Maven Site/Javadoc

Maven coordinates:

<dependency>
    <groupId>biz.paluch.redis</groupId>
    <artifactId>lettuce</artifactId>
    <version>3.0.Beta2</version>
</dependency>

Any feedback is appreciated or create issues on GitHub.

Every then and when, a Java Enterprise project starts. JEE is great. Java Enterprise projects enforce in many cases the usage of the InitialContext and sometimes CDI with using the BeanManager interface. Both are hard to enable within tests.

(more…)

Ever wondered, how to integrate logstash and Tomcat 7 the easy way? Here we go!

(more…)

I’ve released today logstash-gelf 1.4.2, I’ll be available until this evening on Maven Central.

(more…)

Developing applications can be a fun. Developing applications can be also a big pain.

It depends on a lot of factors. Let’s focus today on maintainability/extensibility and a framework concept called Microservice Bus.

(more…)

You might as yourself: Why is there another java redis client? Until now, there are tons available.

True.

Let me tell you about my motivation.

(more…)

I’ve released today logstash-gelf 1.4.1, I’ll be available until this evening on Maven Central.

(more…)

Logstash is one of the hottest tools when it comes to log management. There are lots of integration possibilities in standalone or web applications. But what if you want to capture the whole log output of your JBoss server? Take a look at the snippets.

(more…)

visualizr – Show your data Once in a while I get in touch with lots of time based data, such as monitoring or profiling values.

Calls per minute, actions per time unit and so on. Having such data is a great value but having it visualized gives a different impression instead looking at numbers. I’ve created visualizr. It’s a small Single-Page-Application with a Java-based backend. Visualizr provides support for multiple data sources (a data source can contain multiple charts), auto-reloading and a great user experience.

Give it a try, you can find the code at https://github.com/mp911de/visualizr or as a maven artifact (SNAPSHOT-version until now)

<dependency>
    <groupId>biz.paluch.visualizr</groupId>
    <artifactId>visualizr</artifactId>
    <version>1.0-SNAPSHOT</version>
</dependency>

Sonatype OSS Snapshot Repository: https://oss.sonatype.org/content/repositories/snapshots

It happens currently all the time. I’m always coming back to projects I worked on. Over and over I’m meeting old coworkers and colleagues. Same faces, sometimes a bit older than I left them. This raises mixed feelings. It’s sort of coming home, thus it is different.

Operating a Hadoop cluster means lots of daemons running on lots of machines. When it comes to the logs, and you’re searching for something, it can get nasty, since you do not know, where to search.

(more…)

Today I’ve released the patch 1.2.6 of logstash-gelf. This release contains a performance bugfix which caused an incident in the past. 

The java.util.logging hoghandler’s publish method was synchronized. This caused stalling of different threads which ended up in a denial of service. Usually a restart of the application is sufficient to solve that problem, but that’s only a temporary solution. Other implementations of logstash-gelf (Log4j, Log4j2 and Logback) are not affected.

Links

• Release on Github
• Artifact overview on Maven Central
• Download logstash-gelf 1.2.6

Any feedback is appreciated or create issues on GitHub.

Imagine you're talking about something today with your product owner or boss. It get's the highest priority for today, because it's cool or something. Tomorrow you're not done with it but have again a cool feature idea. Or a collegue of you. And guess what, it gain's again highest priority.

And what about the first topic? Forgotten again. Not having a mid-term or long-term plan can get a horror. Not having a product vision or a strategy for your product is even harder. Every day things can change, old stuff gets obsolete and there is no red thread what your system is and what it's not. How it behaves and how not. This causes uncertainty and the involved developers cannot be confident about their implementation. Always and ongoing discussions and changes of directions are the consequence. The whole product looks and works like a mixed litterbox which causes confused users and no one is happy with the application.

Don't become that team.

I’ve released today logstash-gelf 1.2.0, I’ll be available until this evening on Maven Central.
(more…)

Applications querying datasources can run into it. And perhaps you’ve seen it already: Custom built queries:

(more…)

Since today logstash-gelf supports log4j2. The experimental support allows dynamic field since log4j2 provides a very flexible configuration.

(more…)

Recently I gave logstash a try. logstash is a log management tool (processing, storing). It can push log messages also into elastic search. This setup is ideal for central logging. The easiest way to push log data into logstash seems to me to be gelf (Graylog Extended Log Format). It is a JSON-style format which can be GZIPed using UDP connections. There are a few implementations for GELF using Java (gelfj, gelf4j) which work nicely with Graylog itself. If you try to use them with logstash you’ll run into a few problems, which are caused mainly due the chunk size. gelfj has a fixed chunk size of 1420 bytes. Maven Central contains none of them (only some reassembled artifacts).

(more…)

Lots of companies I know tend to work still in a plain old style. Top-down bosses, either water-fall, ad-hoc or chaos-driven development. Long term release cycles and inefficient, expensive style of development. Continuous Delivery: What’s that, do they ask, why do we need such fancy things?

Many „something as a service“ companies are agile and deliver updates to their services seamless, without any down-time. As soon as they would go down for maintenance a lot of people would stop using it. Imagine Amazon or Google would go offline every day or week for a couple of hours.

In contrast to that, the least other companies go for efficient development, Continuous Delivery and all that other „fancy“ stuff. Some sort of agile development is the most you can expect from german companies. If you’re lucky, they’re doing Test-Driven development or Clean Code, but that’s it. So I started to investigate why this is.

(more…)

Some weeks ago SEMAT hit me: Somewhere Uncle Bob said something about Ivar Jacobson’s book „The Essence of Software Engineering: Applying the SEMAT Kernel“. This was reason enough to sneek peak and after a couple of minutes I bought that book.

(more…)

A couple days ago I wrote about Clean Architecture. I was also excited, how it will look like within code. How does it feel? How can I progress.

(more…)

Java is currently the one and only language having checked exceptions. More and more frameworks switch to unchecked exceptions over time, moving them out of focus, making them invisible. Therefore exceptions become an aspect of modern software and should be treaded in an adequate way: Don’t let your code be exception driven.

(more…)

For quite a long time I was seeking for a reasonable explanation of software architecture. Nobody could provide a sufficient description. Developers and software architects would describe their architecture perhaps this way: We use Model-View-Controller for the Web, EJB as Controller, havin Hibernate as persistence framework.

Layering, multi-tier, Hibernate, Java Server-Faces aren’t an architecture. They are ideas for code organization, tools or somewhat, but for sure not an architecture.

(more…)

Integration Testing describes every test, which integrates more than one component that is tested.

Lots of developers are writing test using JUnit, PHPUnit, NUnit and so on. These tests invoke some piece of code, that do something. This „something“ can be a in-memory calculation, access to databases or even accessing remote services. Unit tests are tests which test a single unit, a method, a class. As soon as this method or class tries to access something, which is outside of your working memory, it’s most likely an integration test. Integration tests are very valuable, but in most cases you perform integration testing without knowing it. In most cases you’ll notice it, as soon as you run your tests without network or somebody kills the database. And then you might be screwed. This example gives some impression what the difficulties are: External systems, services you can’t contol (yet), shared databases. Because of these and sometimes because the benefit is not seen integration tests are not automated.

(more…)

Do you know Puppet? Puppet is a system management/provisioning tooling. It’s intended to specify a set of resources (files, software packages and many more) to be in a certain state. This is very hand for all the cases, where you need for example Apache and PHP, you want to apply an own php.ini, have a customized logrotate.conf, some cron jobs… All this can be specified using Puppet. The first time your system boots, all the specifications are applied and you don’t have to copy config files anymore. Puppet runs also every 30mins (by default) to check for new/changed rules.

(more…)

Some time ago I got in touch with one of my 2006 written applications. I got flashed at the moment I looked into my code.

My intention was to perform some improvements. 2006 had very few frameworks to deal with HTTP interfaces. The most common interface types were GUIs (JSP/JSF) and some SOAP Services. Writing HTTP interfaces for API use (calls from JavaScripts and so on) was still a hard way in Java applications. In most cases servlets were used. Those servlets received either Query- or Form-Params to perform the processing. Sometimes they even accepted/produced some XML. Let me show you some examples and how to solve them using current technology.

(more…)

Prior to this article I wrote about testing SOAP WebServices locally. This was accomplished using JAX-WS. Lots of projects still use Axis or Axis2. Therefore I created a simple Axis2 setup to demonstrate JUnit-Testing of WebServices.
(more…)

In Continuous Delivery you deliver often and to different environments. When it comes to configuration files you have to maintain multiple instances at a time. You can’t just have one fits it all. The Configuration Server is built to support you in this scenario. Basic idea of the Configuration Server is: Serve configuration based on your host for multiple artifacts and versions.

(more…)

As soon as you start Continuous Delivery you don’t want to build artifacts for a specific environment. Once you’ve built and packaged, you want to reuse it in Dev, UAT, Production and so on.
(more…)

I’d guess, nearly every developer, who assisted on releasing and go-live of his software ran into the same issue: How to deal with sensitive configuration data such as passwords, keys and tokens? Hard-coding of passwords it out since the late ’90s. Writing passwords in plain-text into config-files is as well a security breach. Encrypted passwords within config files are much better but demand you to implement some kind of encryption. Still you have then the problem: Where to put the encryption key?

(more…)

It’s a common problem almost which every application has: Where do I put my configuration? Lucky you, rich clients and standalone applications. For most rich client/standalone apps it’s somehow easy, because you have to have somewhere a starter, so you can deploy the config somewhere along the starter. 
(more…)

In my last post about Integrationtest-driven development/Test-driven integration I was talking how to get started with integration testing. Sometimes it comes to the point, where your integration tests need more than only a database and a running system: You have additional dependencies like EJB’s or Web-Services. What now? You can’t always rely on test-systems: Sometimes there aren’t any or only shared instances exist. Sometimes it’s even harder to simulate a specific scenario using real services because of data and logic dependencies.

(more…)

TDD (test-driven development) is a well known development process for now. You start with a idea of a solution and the test. Then you proceed within a cycle of writing production code, then test code again, start over.

(more…)

JUnit test are inherent to almost every professional Java project. Some utilize JUnit more than others, some use it for integration tests. But nearly all of them have similar issues. As soon as they reach the testing boundaries, they have either to take care of expensive test setups or rely on some infrastructure stuff such as databases or remote services. One way is truly keep going on with the known procedure. A different approach is the usage of test dummies. Either as stubs (i.e. sub classing to simulate the desired behavior) or mocking. Mocks are instances of your classes or interfaces which can be told how to behave in a certain scenario.

(more…)

In the early days of development, where technical approaches and feasibilities were depending on “is there any solution available”, lots were happy connecting a database using a web front end. Decoupling and abstraction layers were built to hide the complex persistence API. Ugly web wrappers were also tried to decouple. In the end, you got lots of framework stuff, just because there wasn’t anything else.

Today it evolved. Spring hides and provides lots of functionality. Enterprise Java provides API’s for nearly everything that modern applications use: Persistence, Web control, Web-Services. But somehow, thinking did not change much since then. Today we see tons of mappings, abstractions and decoupling layers. Is it really necessary to build data access objects in case you use JPA? Do you need mappings and tons of layers for exposing database data using a REST API?

It’s worth rethinking. Just because we always did isn’t the right answer.

MongoDB is known for its easy and near zeroconf setup. But when it comes to the point, to setup a sharded cluster with replication, things can get complex. Therefore I’d like to provide a configuration tooling to setup such a cluster with only a couple of simple commands.

(more…)

Some time ago, I’ve used an USB-based relay to control a XFD (extreme feedback device). Now, let’g go a step beyond that and let a relay dance.

(more…)

I’m working for 3 years now with an agile approach. Before that, it was either sort of waterfall or a complete mess. The first agile steps were as well sort of agile but not the real one.

My experience is, that the way my current team is working, is real agile for about a year now. Before that, it was a try and error from which we learned a lot.
(more…)

In every evolution step has a start and an end. Sometimes they’re blurry, so you can’t say for sure, where you are at. In every step, sooner or later, arises the question: What’s next? What comes after agile? What ist the future of agile? This is today exactly my point.

(more…)

A couple months ago, the team, I work on, introduced integration tests. Before that, we had only regular Unit tests, which ran in a very isolated environment. You do not have a control, how your modules behave when they interact in a real environment. You cannot test a lifecycle with persistence to be confident, because you need much more than mocks.

(more…)

Recently I reported about Clean Code Development. Everybody who’s visiting trainings, courses and sessions is amazed. It’s easy, most topics are obvious but not everyone is always aware of these topics. But when you look deeper constrained by the big picture there is as well a dark side.

(more…)

In my last post, Unit-Testing of Web-Services with JUnit – SOAP Services, I’ve demonstrated how you can test local SOAP Services with remoting in a local Unit-Test. But since SOAP service become more and more unpopular, there might be a different technique. Currently REST becomes the name of the game. So let’s take a look at JUnit testing RESTful services with remoting.

(more…)

I want to give you a better insight in Clean Code. Therefore i created a bunch of code lines with examples. These show you a set of good/bad (after/before) scenarios. You can find these at GitHub:

https://github.com/mp911de/CCD

Now it’s 6 years ago, since I started to apply Clean Code Development (CCD). For me it’s today way more than clean-code-developer.com. It became a system of values, based on best practices, principles and a set of pragmatism.
(more…)

There are days or weeks, you feel, every day something is failing. Crashing systems, strange bugs, ghosts. For some reason you don’t know, why this happens, over and over. But you are the man in the middle. You’re the one, having all those dependencies. Then you have to fix it.
(more…)

With modern Tooling it’s easy, to Mock a WebService – take your Service-Class, inject Mocks (for your dependencies) and tell the Mocks how to behave. The next usual Stage is Integration testing. That one can become hard, you have to rely on remote services. Once the remote service is down, you even can’t test the webService remoting. Does the stack behave in the right way? How about the WSDL? Is it right?
(more…)

Roy T. Fielding wrote and described REST in his dissertation in the 2000’s. He basically told about the an architectural pattern and mapped it to most common protocol: HTTP

Since these days the IT community began to realize and to understand more and more what it means to deal with resources, what power and strength HTTP provides. But what’s so special about it? And why does everybody talk about it but the least understood it? Why is today everything called REST, but in fact it’s not REST? Let’s dig into it.
(more…)

Exceptions, yes, what are exceptions? Exceptions are situations, where the planned flow cannot be completed anymore. So they are exactly the same as dealing with business logic. Exceptions are only a more elegant mechanism. They allows us to provide in an emergency case (unexpected parameters, errors in file access) an adequate response.
(more…)

In many software systems and use cases there are exceptions. Exceptions, that are triggered by conditions that do not allow the expected sequence (ideally). These exceptions and dealing with those are hiding behind the lapidary used “exception handling”. A fast-created try-catch is often not the solution. And in those cases in which exceptions are forwarded to the client, there are some pitfalls.

(more…)

In vielen Software-Systemen und Use Cases gibt es Ausnahmen. Ausnahmen, die durch Zustände ausgelöst werden, die nicht den erwarteten Ablauf (Idealfall) ermöglichen. Diese Ausnahmen und der Umgang verbirgt sich hinter dem lapidar benutzten “Exception Handling”. Dabei ist ein schnell erstellter try-catch oftmals nicht die Lösung. Und in den Fällen, in denen Exceptions an den Client weitergeleitet werden, gibt es einige Fallstricke.
(more…)

Vor ca. einer Woche habe ich den Quellcode meines JBoss Development Deployers auf GitHub migriert. Das Repository ist unter https://github.com/mp911de/JBoss-Development-Deployer erreichbar. Happy social coding!

Erst neulich habe ich meine Server konsolidiert und virtualisiert. Aus 3 Root-Servern habe ich zu einem größeren Root-Server mit virtualisierten Sub-Systemen zusammengeführt.

(more…)

Häufig wird die Anforderung gestellt, Aufgaben im Hintergrund zu verarbeiten. Durch den Trend der Webanwendungen trifft früher oder später diese Anforderung auch auf ein Web-Projekt zu. Erst kürzlich kam dieselbe Frage in der JBoss Community auf.

(more…)

Vor nicht allzulanger Zeit, da gab es mal ein Projekt, in einem nicht weit entfernten Land. Und was jetzt folgt ist keine Gute-Nacht-Geschichte. In einem meiner Projekte gab es folgendes Setup: JBoss, Hibernate mit Cache und einen Cluster

Ein regulärer Stack für Enterprise-Projekte. Das System funktionierte zuverlässig und führte in regelmäßigen Abständen zu unterschiedlichen Daten auf den verschiedenen Nodes (bei den selben Objekten)

(more…)

Im ersten Teil ging es um die Top 5 Ursachen für Performance-Probleme in Java und J2EE-Umgebungen. Nun will ich mich den anderen Ursachen widmen, die zu Lasten der Performance gehen. 

(more…)

Das Release 0.6 des JBoss Development Deployers ist nun da. Folgende Features sind nun enthalten:

• EAR Deployment mit flexiblem Class-Path und modulweiser Umleitung in Entwicklungsverzeichnisse
• WAR Deployment mit flexiblem Class-Path und Umleitung ins Entwicklungsverzeichnis

Der JBoss Development Deployer ist unter http://jb-dev-deploy.sourceforge.net/ verfügbar.

Demnächst ist es soweit. Das Release 0.6 des JBoss Development Deployers kommt im Mai. Das größte Feature ist das EAR-Deployment. In der EAR-Datei können die Pfad zur Entwicklungsumgebung angegeben werden und damit sind Redeployments oder schwierige JSP-Bearbeitungen im Temp-Verzeichniss des JBoss Vergangenheit. Derzeit befindet sich das Projekt noch in der Testphase, aber es hat zahlreiche Tests mit EJB, WAR (mit unteranderem JSF-Komponenten) und weiteren Modulen bestanden.

Der JBoss Development Deployer ist unter http://jb-dev-deploy.sourceforge.net/ verfügbar.

Eine häufige Aussage von Anwendern ist: Schon wieder ist alles langsamer geworden. Oftmals nach neuen Releases, bei denen viele neue Features hinzugefügt und bestehende Funktionen erweitert wurden.

(more…)

EMF kann aus einem Modell Java-Code erzeugen. Das so erzeugte Java-Programm kann Instanzen dieses Modells erstellen, abfragen, manipulieren, serialisieren (eingebaut als XMI oder anderes XML, mit Plugin auch in einer relationalen DB), validieren und auf Änderungen überwachen (für MVC). Darüber hinaus wird JUnit-Code erzeugt, der den generierten Code testet.

(more…)

Wenn ich heute jemanden Frage, was er in der letzten Zeit so gemacht hat, so antworten über 90 % “Ein Projekt im Bereich …”. Dabei sind dies bei weitem nicht alles Projekte im IT Bereich. Ja die Projektarbeit dringt in immer weitere Bereiche von Unternehmen vor. So stellt sich mir immer wieder die Frage: Wann ist ein Projekt erfolgreich, wann ist es fehlgeschlagen?

(more…)

Viele Entwickler, denen ihre Software wichtig ist, haben eine Idealvorstellung vom Prozess der Entwicklung.

(more…)

Entwickler in Multi-Modul-Projekten haben viele Aufgaben. Während der Entwicklung wirkt sich besonders das Deployment/Re-Deployment einer Anwendung negativ auf die Effizienz aus. Es gibt zwar kostenpflichtige Tools (z. B. JRebel), die Abhilfe schaffen, jedoch noch kein Open Source Tool.

Dies habe ich zum Anlass genommen und einen Development Deployer für JBoss AS entwickelt. Mit diesem Deployer können J2EE Multi-Modul-Projekte während der Entwicklung in der IDE bleiben, das lästige JAR’s bauen/deployen entfällt. So können beispielsweise in einer Web-Anwendung (WAR-Deployment) mehrere Klassenpfade (z. B. classes- und/oder lib-Verzeichnisse) konfiguriert werden.

Das Projekt wird bei SourceForge gehostet und ist unter http://jb-dev-deploy.sourceforge.net/ erreichbar.

Derzeit wird ein Web-Deployment (WAR) unterstützt (getestet mit BouncyCastle/JSF-Komponenten). Geplant ist die komplette Unterstützung von J2EE Projekten (EAR/SAR/HAR).

Ich freue mich auf Feedback.

In nahezu allen Enterprise-Projekten sind Zugriffe auf Datenbestände in Datenbanken erforderlich. Manchmal direkt über JDBC oder, in den besser organisierten Projekten, über objektrelationale Mapper wie JPA, Hibernate oder ähnliche.

(more…)

Seit dem 25.06.2010 ist es soweit: Mein Artikel Code-Recycling: Refaktorisierung einer Alt-Anwendung in der Praxis (Refactoring) wurde in der OBJEKTSpektrum veröffentlicht. Ich bin auf Feedback gespannt und freue mich über andere Erfahrungsberichte mit Refactoring.

Was kann man mit dem Eclipse Modeling Framework so anstellen?. Nun, zum Beispiel statt das Modell in XML zu serialisieren, können die Daten auch in einer Datenbank abgespeichert werden.

(more…)

Wer kennt Sie noch nicht? Alle namhaften Reinigungsmittelhersteller bewerben ihre Zauberschwämme und Schmutzradierer. “Ein neuartiger Schwamm, der alle Verunreinigungen beseitigt!” So oder ähnlich lauten die Versprechen. Und das schlimme daran ist: Es stimmt!

(more…)