Switching from Proxmox 3.4 to 4.0

I have a few virtual servers on a server with proxmox. It’s perfect for doing some simple tests without bad surprises like I had with AWS.

Proxmox 3.4 was working fine. The only issue was its old 2.6 kernel. It was preventing me from using docker. So I switched to Proxmox 4.0 to get the brand new 4.2 linux kernel.

So here are some issues I discovered:

  • OpenVZ containers had to be converted to LXC ones without any advantage and one big inconvenient: There are not saved on some actual files but into one big virtual disk file. It prevents you from extending and reducing disk space allocated to your instances (which could be done at any time from 3.4). It makes remote backup harder, you can’t just sync files with rsync. Solutions like rsnapshot cannot be used on the host system.
  • The system is a lot slower. You can see the graph below of before and after the upgrade.
    screenshot-ovh5.webingenia.com 8006 2015-10-25 22-15-31

Cassandra on droid.io

droid.io is an other travis clone.

I gave it a try to execute some automated tests on top of cassandra. Unfortunately it doesn’t support cassandra out of the box. But adding support for it is in fact quite easy:

Here is a script to load cassandra and wait for its startup:

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/jre
curl -LO http://archive.apache.org/dist/cassandra/2.0.5/apache-cassandra-2.0.5-bin.tar.gz
tar xf apache-cassandra-2.0.5-bin.tar.gz
ln -s apache-cassandra-2.0.5 cassandra
cd cassandra
sudo mkdir -p /var/lib/cassandra /var/log/cassandra
sudo chown `whoami` /var/lib/cassandra /var/log/cassandra
for i in {0..30}; do echo "Waiting server ($i)..." ; nc localhost 9042 </dev/null && exit 0 ; sleep 1; done;
exit 1

Let’s GO

Why should you spend time learning go ?

The language is great

  • Simple: Languages syntax is quite small and easy to learn. Complex features that other languages provide have been omitted: inheritance, generics and pointer arithmetic.
  • Elegant: No semicolons. The scope of variables has been thoughtfully designed, it’s a huge win for readability of code. I personally add (often useless) brackets in my C to make it more readable. The defer keyword allows some clean and simple resource freeing.
  • Easy to use: Having a garbage collector makes it easy to use.
  • Built for concurrency: Making concurrent code has never been so easy. All IO calls are blocking but in a go-routine kind of way, meaning that as soon an I/O or time operation occurs, the current running thread will execute an other go-routime. It’s the java’s Future mechanism made easy.
  • Comes batteries included: Like python, it has some great standard packages.
  • Go on C libs: cgo makes it simple to integrate C code and C libraries. But for all the I/O related task, go internal code will perform much better (it shares a threads).
  • Unicode support: It’s one thing you won’t have to worry about.

But it’s not just the language

It is given a complete environment. You can can type “go...

  • build: Builds your code. No use for a Makefile or complex stuff like that. “go build” takes care of everything.
  • run: Runs your code if you don’t want to build it.
  • test: Alows some very easy (and fast) unit testing of your code.
  • get: Fetches any kind of package.
  • fmt: Formats your code. You don’t have to define some code formatting rules. Your code is reformated following the go requirements.
  • doc: Documents your code. No doxygen or javadoc, go has it.
  • vet: It inspects your code to help you detect wrong behaviors.
  • fix: Fixes your code by replacing old APIs calls by new ones.
  • install: Installs your packages (libraries). Installing a package on a host is easy as hell.
  • tool: Contains a set of useful tools. Most important one in my opinion is the pprof CPU/memory profiler.


To be exhaustive, I’d like to mention few issues you can expect from this languages at this stage.

  • Huge binaries: Mostly because produced go binaries are almost completely static but not only, they are huge (like 2MB big hello world). The problem is known and likely to be reduced in the future.
  • No big GUI libraries: The language is still young (Go 1.0 was released in Mars 2012) and focus has been put on server side code. So there’s not much librairies or librairies environment built out there.
  • No real localization: In python, we usually use gettext with _('stuff') but here we don’t. In Go you have solutions, but none of them is provided by the standard packages at this stage.
  • You might be disturbed by Go’s interfaces: If you come from java with its interfaces inheritances and its powerful namespaces. You might be disturbed that some interfaces included other interfaces because they include their functions.
  • Depencies are not versionned. When you use imports within your code, it might not work in a few months. Also there are ways around it. Coming from java’s maven or gradle, it’s quite disturbing.

Can we bet on it ?

Is it matture enough ? Is is a real disruption ? Will the language actually take-off ?
It’s definitely a major evolution. Because you have many good and exclusive feature while sill having some good old C language bindings and a LOT of libraries have already been ported or binded on it. The "go get" command is also a great recipe for boosting community support.

Getting started

To get you started, I would recommend making the golang tour, then installing go and using a good IDE like LiteIDE or SublimeText + GoBlime.

If (or when should I say) you are hooked, read the effective go page. There are many things like the getter/setter conventions that you will only see here or in the thousands of open-source Go projects outhere.

Just as a quick note: It’s static language. If you come from PHP, ruby or python with little C/C++/java experience, you might find the variable scoping a little bit hard to understand at first.

If you want to start making webapps, I can recommend trying revel or martini (+ fresh for automatic reloading)


I’ll try to update this page later.

To all my dear pure C developers

I’ve spent quite some time integrating a lot of C code from different people to turn it into production-ready software. It is surprisingly interesting, but the first few days are usually quite painful for these reasons:


Many C developers have done hundreds of projects, they always need logging but when an error occurred they still like to do some standard fprintf(stderr, ... );. I use something similar to this. But you should at least insert the __FILE__ and __LINE__ macros. It will save lots of people hours of debugging.

Use the warnings

On gcc, you should always have -Wall -Werror (and -Wextra when possible) enabled. This will avoid a lot of bugs.

Error code

Do not reinvent your own error codes. Use the the POSIX error codes.

Error handling (and logging)

Check the return codes, log the abnormal ones.
Sample code that can make you lose quite some time:

FILE * f = fopen("logs.txt","w");
if ( ! f )
   goto end;

THIS IS WRONG, we should have something like this:

LL_WARNING("Could not open log file !");

using this kind of macro.

Dirty numbers

You know how to count in bytes, how pointer and offsets work, that’s great. But you should also know that you can always replace this by a sizeof(). That way, we won’t have to be as smart as you to read your code (and fix it).

Great POSIX standard headers and libraries

The standard errors code already cover a lot of cases you could encounter.
You can use booleans including stdbool.h, it’s even standard.
Argument parsing is standard, don’t make your own strange argument parsing code that won’t even (fully) document.

Great non standard libraries

Like json-c to serialize your data, or zeromq to handle network communication.

Bad memory management

Leaks and buffer overflow can happen. And so as many other issues out there. This is why you should use valgrind frequently.

“It worked fine on my computer” is never a good excuse, but on C it makes lot less sense. There are many memory mistakes that you will miss on a 64bits Linux with 8GB of RAM and that you will definitely see on a 32bits Linux with 128MB of RAM (which is high-end in the embedded world).

Malloc is not your (only) friend

In most cases, allocating things on the stack is enough. A lot of developers seem to forget it. And it’s usually the same who forget to free their mallocated pointers.

A lot of people seem to forget that you have the right to allocate arrays dynamically on the stack. Declaring an array like that is ok:

uint8_t bin[strlen(hex)/2];

Use a debugger

Debugger will make you save a lot of time especially when they are well integrated in IDEs like netbeans. You can just immediately know why something crashed or why the programmed stalled for no apparent reasons.

Tiny note on debugging: If you have strange issues and can’t seem to get a correct stacktrace, it’s probably a stack-smashing issue. Compile your code with the -fstack-protector flag, this will report the stack smashing (most probably a buffer overflow) as soon as it occurs.


In most of the cases, your compiler will do a great job. Don’t use inline, prefetching or any other crazy optimization unless you really know what you do.

Don’t re-create unreadable network protocols

Use something simple readable like JSON over zeromq. It’s easier to create, debug, monitor. Please don’t create a new binary protocol.

Learn other languages

There are many scenarios where using something else (like python or go) will give you a huge productivity boost, it will probably run slower and produce bigger binaries but in most of the cases it really doesn’t matter. When making network servers, python, go, node.js (java in some ways) make the job a lot easier.


Use standard tools like autoconf, automake or create a simple and easy to maintain Makefile like this one.


Spend the time to learn how to use a an IDE like netbeans or take the full advantage of vim.

Cassandra CQL3 internal data structure

I’m a huge fan of cassandra, I’ve been playing with it since 0.7 and I’ve never stopped using it. It would say it’s most amazing features are: Always working and simple replication + predictable performances.

I was very happy when it went from a key-value store to a well structured database with CQL. With CQL you can focus on your data, and less on how you should organize your own structure to handle it properly. Still, behind the wheels, it works the same (it’s still a KV store). That’s why it’s very important to understand how the internal structure is done:

it’s not a perfect replacement yet. For example, you can’t get the collection elements timestamp (called writetime in CQL). “SELECT map[‘value’] FROM table;” doesn’t exist (not CQL compatible), so “SELECT writetime( map[‘value’] ) FROM table;” doesn’t either unfortunately.

This problem is known by Cassandra’s dev team but there’s indeed a syntax issue to solve first.

Pieter Hintjens is a smart guy

I’m very interested by everything around internet of things technologies (M2M). And the components I like to follow closely are the distributed databases which allow to make timeseries without single point of failures like cassandra and hbase and the message broker like RabbitMQ and ZeroMQ.

This little video isn’t really about ZeroMQ, it’s more about how build a community around a project. Personal creativity is great thing, we can all experience it some way or an other. Creating in a team gives you some new perspectives on how you can make better things (and, usually, be a better person). But creating as part of a community of people you don’t known and share your ideas/experiments/knownledges/insights with them is amazing. How to compile everyone’s mind into something better.
I’ve personnaly had the best fun building things that evolved with time, on a daily basis. You know the agile methodology when you use it for real.

My favorite part was:

We discovered, by trial and errors, – as always, it’s always by making mistakes and see what goes wrong – that smart people make big mistakes. And the smarter you are, the more you make big mistakes.

But the rest of the talk is also great. Around 32:00, he talks about how he prefers to merge bad code to let people engage with the community and become part of it then passing on them (and their dumb code). It’s definitely valuable to any kind of orgzniation trying to build communities around quality content. It’s always better to let people engage, even in a bad way, it makes other people engage as well and then you have to build the tools so that the outcome always is positive (you want a good community around/AND a good content in the end).

You can do any kind of project with any language

Our language helps us express our ideas and emotions. Computer science languages help us build things following our ideas.

There are hundreds of computer science languages currently available to help do pretty much any kind of work. They were all built because (at least) one person once thought “this language isn’t good enough”.

I often like to compare the capabilities of languages like C/C++, C#(.Net)/java, scala/python/ruby, javascript/coffeescript, etc. And I’m always amazed how the smallest changes that were brought to existing languages allowed to gain in efficiency and clarity.

And yet, I can still find people telling me: But you can do this in this language as well.

Deploying your maven web apps on glassfish with jenkins

Simple deployment

If your jenkins server is installed on the same host as the glassfish server it’s quite simple.
You just have to let maven do its thing and then add this shell command:

/usr/local/glassfish/bin/asadmin --echo=true --host=localhost --port=4848 --user=admin --passwordfile=/secure/place/for/passwords/domain1_password --secure=false deploy --force=true --name=myproject --contextroot=/myproject target/*.war

The /secure/place/for/passwords/domain1_password should contain this:


Deploying different versions

With parametrized build you can go a little further and decide, for instance, if you want to deploy the current version as the production one or not. You can switch the build as a parametrized one and add a “stable” parameter (with a default value of false) for example.

The the only thing you have to do is to switch the previous code to:

if $stable; then
    /usr/local/glassfish/bin/asadmin --echo=true --host=localhost --port=4848 --user=admin --passwordfile=/secure/place/for/passwords/domain1_password --secure=false deploy --force=true --name=myproject --contextroot=/myproject target/*.war
    /usr/local/glassfish/bin/asadmin --echo=true --host=localhost --port=4848 --user=admin --passwordfile=/secure/place/for/passwords/domain1_password --secure=false deploy --force=true --name=myproject-test --contextroot=/myproject-test target/*.war

Using the jabber plugin

Then you can use the jabber plugin and you will have interesting conversations with your jenkins build system, like:
!build myproject

And when you feel you are ready to go live:
!build myproject stable=true


  • It’s even simpler with other servers like tomcat or jetty.
  • Glassfish is often considered as very heavy but as most of the OSGi based softwares, you can remove a lot of its parts and make it as light as tomcat.

Valgrind might be wrong

Sometimes valgrind can report the right problem with the wrong stack trace:

make && sudo valgrind --db-attach=yes --leak-check=full --track-origins=yes --show-reachable=yes ./dist/Debug/cbadaemon 
==4423== Memcheck, a memory error detector
==4423== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==4423== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==4423== Command: ./dist/Debug/cbadaemon
==4423== Conditional jump or move depends on uninitialised value(s)
==4423==    at 0x4922C47: __pthread_mutex_unlock_usercnt (pthread_mutex_unlock.c:38)
==4423==    by 0x804E91A: ServerContext::ServerContext() (ServerContext.cpp:11)
==4423==    by 0x804EB92: ServerCore::ServerCore(int, char**) (ServerCore.cpp:15)
==4423==    by 0x804F627: main (main.cpp:57)
==4423==  Uninitialised value was created by a heap allocation
==4423==    at 0x48D273C: operator new(unsigned int) (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==4423==    by 0x804F60F: main (main.cpp:57)
==4423== ---- Attach to debugger ? --- [Return/N/n/Y/y/C/c] ---- y
(gdb) bt
#0  0x04922c47 in __pthread_mutex_unlock_usercnt (mutex=0x75e80d8, decr=1) at pthread_mutex_unlock.c:38
#1  0x0805d15b in RequestManager::RequestManager (this=0x75e8030) at src/transac/RequestManager.cpp:27
#2  0x0804e91b in ServerContext::ServerContext (this=0x75e802c) at src/server/ServerContext.cpp:11
#3  0x0804eb93 in ServerCore::ServerCore (this=0x75e8028, argc=1, argv=0xfef79b94) at src/server/ServerCore.cpp:15
#4  0x0804f628 in main (argc=1, argv=0xfef79b94) at src/main.cpp:57
(gdb) q

And it’s not an optimization problem as it was compiled with:

-c -m32 -g -rdynamic -O0 -fPIC -Wall -Werror