Florent Clairambault

On my spare time, I manage a handful of servers. And even if it’s not really my job, I try to do it well and efficiently. All of them work on Debian because it’s simple to manage. I started using cron-apt a few years ago. I started by upgrading everything automatically, this was a big mistake. I switched to only sending mails on available upgrades and doing the upgrade manually. But this is also quite painful because 95% of the time, it consists in typing “apt-get dist-upgrade -y” and waiting. And have no free time for doing stupid things.

So here is my cron-apt configuration, I like it a lot:

In /etc/apt:
- I removed the sources.list file
- I put the content of my sources.list into sources.list.d/main.list, it should look something like that:

1
2

deb http://http.us.debian.org/debian stable main contrib non-free
deb-src http://http.us.debian.org/debian stable main contrib non-free

- I created a directory sources.security.list.d
- I put the following content:

1
2

deb http://security.debian.org/ stable/updates main contrib non-free
deb-src http://security.debian.org/ stable/updates main contrib non-free

Then I added the repositories with packages I want to manually upgrade to /etc/apt/sources.list.d/ and the ones that I want to automatically upgrade (which means that they can’t require any user interaction) to /etc/apt/sources.security.list.d/.

The interesting part is here, in /etc/cron-apt/action.d, this what I have:

0-update

1
2

update -o quiet=2
update -o quiet=2 -o Dir::Etc::sourceparts=/etc/apt/sources.security.list.d -o Dir::State::lists="security-lists"

We launch an update of the two kinds of repositories. For the sources.security.list.d one, we use also a different Dir::State::lists parameter (which is the directory the cache file) so that we don’t to re-download the content of the index files every time.

2-install-security

1

dist-upgrade -y -o quiet=1 -o Dir::Etc::sourceparts=/etc/apt/sources.security.list.d -o Dir::State::lists="security-lists"

We launch the upgrade (dist-upgrade actually) only on the repositories defined in /etc/apt/sources.security.list.d.

3-download

1

dist-upgrade -d -y -o APT::Get::Show-Upgraded=true

Then we only download files for the upgrade of the non-security packets.

6-clean

1

autoclean -y

And we finally delete all the old packets (the ones that will never be used).

If you want to play with the apt settings yourself, you should use apt-config to see what can change to fit your needs.

I’ve been testing btrfs for some months now. One of the most interesting features of this file-system is its snapshoting capabilities. Before that I was using rsnapshot. The issue with rsnapshot is that its lowest atomic level for snapshotting is the files themselves using hard-links. So any database table where one row is changed is copied completely. Btrfs as you might guess will only copy the modified chunks (I don’t know the atomicity of them [but who cares?]).

Here is a simple I’ve been using during these last months to backup my laptop and (remotely hosted) servers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139

#!/bin/sh
 
# === ARGUMENTS PARSING ===
 
# We don't want to define a default period
PERIOD=
 
while echo $1 | grep ^- > /dev/null; do
 
    if [ "$1" = "--daily" ]; then
        PERIOD=daily
    fi
 
    if [ "$1" = "--monthly" ]; then
        PERIOD=monthly
    fi
 
    if [ "$1" = "--period" ]; then
        PERIOD=$2
        shift
    fi
 
    shift
done
 
if [ "${PERIOD}" = "" ]; then
        echo "You have to define a period  with the --period arg !" >&2
        exit 1
fi
 
# === END OF ARGUMENTS PARSING ===
 
# === PARAMETERS ===
 
# * Device we will use
DISK=/mnt/externe3
 
# * Subvolume used for the backup
SUBVOLUME=${DISK}/servers-backup
 
# * Current date (you could limit the date to +%Y-%m-%d)
DATE=`/bin/date +%Y-%m-%d_%H-%M-%S`
 
# * snapshot directory that will be used
SNAPDIR=${DISK}/snap/servers-backup
 
# * snapshot volume that will be used
SNAPVOL=${SNAPDIR}/${PERIOD}-${DATE}
 
# * max days to keep daily backups
MAX_DAYLY=60
 
# * max days to keep monthly backups
MAX_MONTHLY=365
 
# * Alert limit
LIMIT_ALERT=95
 
# * High limit
LIMIT_HIGH=90
 
# * Low limit
LIMIT_LOW=85
 
# === END OF PARAMETERS ===
 
# We get the space used over the total allocated space and the total percentage use.
# This is NOT the device total size but it's a lot more reliable than "df -h"
DISK_USED=`/sbin/btrfs filesystem df ${DISK}|grep Data|grep -Po "used=([0-9]*)"|cut -d= -f2`
DISK_TOTAL=`/sbin/btrfs filesystem df ${DISK}|grep Data|grep -Po "total=([0-9]*)"|cut -d= -f2`
DISK_PERC=`echo 100*${DISK_USED}/${DISK_TOTAL}|bc`
 
# We create the snapshot dir if it doesn't exist
if [ ! -d ${SNAPDIR} ]; then
        mkdir -p ${SNAPDIR}
fi
 
cd ${SNAPDIR}
 
# If we are over the low free space limit,
# we delete two days of daily backup.
if [ $DISK_PERC -gt $LIMIT_LOW ]; then
        echo "LOW LIMIT reached: $DISK_PERC > $LIMIT_LOW : Deleting 2 days" >&2
 
        OLDEST_FILES=`ls --sort=time -r|grep "daily-.*"|head -2`
        for file in $OLDEST_FILES; do
                /sbin/btrfs subvolume delete $file;
        done
 
fi
 
# If we are over the high free space limit,
# we delete a month of monthly backup
if [ $DISK_PERC -gt $LIMIT_HIGH ]; then
        echo "HIGH LIMIT reached: $DISK_PERC > $LIMIT_HIGH : Deleting 1 month" >&2
 
        OLDEST_FILES=`ls --sort=time -r|grep "monthly-.*"|head -1`
        for file in $OLDEST_FILES; do
                /sbin/btrfs subvolume delete $file;
        done
 
fi
 
# If we are over the alert free space limit,
# we delete the first two oldest files we can find
if [ $DISK_PERC -gt $LIMIT_ALERT ]; then
        echo "ALERT LIMIT reached: $DISK_PERC > $LIMIT_ALERT : Deleting the 2 oldest" >&2
 
        OLDEST_FILES=`ls --sort=time -r|head -2`
        for file in $OLDEST_FILES; do
                /sbin/btrfs subvolume delete $file;
        done
fi
 
 
# We touch the subvolume to change the modification date
touch ${SUBVOLUME}
 
# We do a snapshot of the subvolume
if [ ! -d "${SNAPVOL}" ]; then
        /sbin/btrfs subvolume snapshot ${SUBVOLUME} ${SNAPVOL}
fi
 
# We delete the backups older than MAX_DAYLY
find ${SNAPDIR} -mindepth 1 -maxdepth 1 -mtime +${MAX_DAYLY} -name "daily-*" -exec /sbin/btrfs subvolume delete {} \;
 
# We delete the backups older than MAX_MONTHLY
find ${SNAPDIR} -mindepth 1 -maxdepth 1 -mtime +${MAX_MONTHLY} -name "monthly-*" -exec /sbin/btrfs subvolume delete {} \;
 
 
# This is the actual backup code
# You need to save your data into the ${SUBVOLUME} directory
 
# We will only do the actual backup for the daily task
if [ "${PERIOD}" = "daily" ]; then
 
rsync -auv /usr/local/bin ${SUBVOLUME}/localhost/usr/local
 
fi

Then this is how you can use it by adding these cron-tasks :

1
2

0 12 * * *  user /usr/local/bin/backup-servers --period daily   >/var/log/backup-servers-daily.log
55 10 1 * * user /usr/local/bin/backup-servers --period monthly >/var/log/backup-servers-monthly.log

I bought the Kindle. It might sound strange considering I already have an iPad, but I couldn’t help myself to buy a “e-ink” enabled device.

The user experience around the Kindle is OK but not great, because it’s not a touch screen. The real amazing thing is the sync between amazon, the kindle and all the other possible devices.

• There’s the sync when you buy a book. It appears instantly on the device. This feeling is so great. You do the “one click buy” and plop, you can read it. I know you can download most of the ebooks for free. But it takes at least 10 times more time. And I think we should prefer to download things legally when it’s at a fair price and faster to download than illegally.
• And there’s the sync between your devices: I can read the same book on my iPad (better at night), my iPhone (I always have it) or the kindle. And every time I open any of these devices, it tells me “You were last reading on page X on this device, do you want to go there ?”.

So thank you amazon for making a better world.

On a side node, you should know that you can’t cancel ebooks you bought on amazon, even with the “one click buy”. You have to send them a mail to ask for the cancellation of your order. But every time I contacted them they replied within 20 minutes, even at 2 AM.

Same title. I wish I was more original.

I’ve been missing writing on this blog. It’s very fulfilling to write stuff around subjects we love.

What’s new in this document ?

• Netbeans 7.0 with the TC65i
• The use of pre-processor
• Small thougths around the EGS5.
• Personnal advices on project/product management

This can be considered as a draft.
I’m waiting for your comments, even (or especially) negative ones, to help me improve this document.

Hi everyone,

I updated the TC65Dev document. Mostly because there’s a good chance I will only have less free time. I modified few parts and added some new content. It is now 50 pages long, this is a lot more than I originally intended to do.

Click here to get it

I just tested the sharing of the iPhone 3G internet connection that comes with iOS 4.3. And it works great. I think Apple figured out that sharing 3G over WiFi was the main reason for jailbreaking iPhones.

Until now I was wondering if I would take the 3G or Wifi iPad 2. Now I’m pretty sure I will take the WiFi version. The good news here is also that Bouygues doesn’t seem to lock it (I accepted the mobile provider settings from the iTunes just after I updated the iPhone).

Sorry for the blog inactivity, I’m quite busy. And thank you for being so many (4000 /month in Analytics) to come here for reasons I can’t really explain (except I’m a very interesting person).

Debian released a new version of their system. I updated it on the server that powers this blog, it took me something like one hour to do the whole system upgrade. There was only a little glitch with mysql’s my.cnf file that had an unsupported “skip-bdb” line. Everything else went fine…

The very good thing in this new release is the new kfreebsd version (available in i386 and x86_64). It brings the power of the FreeBSD kernel to the great Debian OS. If you don’t see the point, read this. To put in a nutshell: a more stable kernel with less legal issues, better vendors support and the same softwares.

The cloud

Just in case, you still haven’t understood, the future is in the clouds. The more I use online product, the more I feel it. I’m now typing all my documents into google docs. Ok, I copy/paste them to Word for my retarded friends that live bellow the clouds, but soon I won’t. Even the important things that I have to do on actual files are synced by Dropbox (still on the cloud).

Webkit

Have you noticed that WebKit, the engine behind the Google Chrome browser also powers the Apple’s iOS (iPod Touch, iPhone, iPad), Google’s Android phones, RIM’s BlackBerry, Nokia’s phones and our french Freebox v6. So basically all the current phones and tablets. You can build web apps and make sure they will work with all the current desktop browsers and phones.
But wait, wasn’t it the idea behind the J2ME. And wasn’t it a total failure? Not really, because J2ME is about phone, WebKit web apps are about every connected things (there’s no reason to stop at computers, tablets and phones, advertising space would be much easier to handle with browsers). Porting J2ME to one environment to an other was painful, with a web browser it’s much more simple.

Google Chrome OS

I just saw a facebook friend wall page:

BTW, I applied too.

So Google actually started his Google Chrome computer test program.

There’s just one thing I don’t understand here. The iPad (I’ll buy the next version) is centered on the usage: Your read your mails, play and stay informed on your couch. Android is about making better phones to reduce the gaps between the iPhone and the stupid phone constructors OS. Desktop computers are here, well because we sometimes do have to work.

But what is Google Chrome OS about? Will we be able to use it on our couch? will we be able to work with it? Because it’s clearly centered on the new Chrome Web Apps Store. Is it about consuming or producing content?

If you are wondering how do perform the Chrome apps in the real world: VERY WELL. Take a look the Chrome Web store with the Chrome 9 browser (currently bêta), you can take the last fm music player or test a graphically amazing 3D app., cooliris:

Note:
History of WebKit

Hi the Cinterion community,

Cinterion France asked me to remove all the Cinterion TC65 / TC65i development materials I published on my blog. The good news is you can contact the Cinterion office of your country and sign a NDA to get the latest version of their manuals and SDK.

Hi the TC65 community,

There’s a new software in town, after JOBexFTP from Ricardo Schmidt, Christoph Vilsmeier decided to create the TC65SH tool which apparently does the same thing but faster.

Best regards,