One weird bug on the XT75

WARNING: All the Cinterion related content from this blog will be removed to go to the javacint wiki soon. Please get used to going there.

You might have faced this error with your X765 chip. The program crashes with this error :

1
2
3
^EXIT 00010000,02d6414253485f4c434c2c70726f6365647572655265636f72645f702d3e6e756d6265724f66526567697374657265645461736b73203c3d204d41585f4e4f5f524547495354455245445f5441534b53
 
^SHUTDOWN

If you convert the hex array to some text, that will give you :

1
#ABSH_LCL,procedureRecord_p->numberOfRegisteredTasks <= MAX_NO_REGISTERED_TASKS

Which might mean something to someone. But the point is, you won’t find any help. Mostly because it’s an uncommon error.

The error comes from the GPRS connection management. I had it when I was using a wrong APN (with the “AT^SJNET” command) to connect to a host. In my program, connection failed with a classic IOException (“Profile not found”) but 1 or 2 minutes later, the chip was ALWAYS crashed (with the “^EXIT” URC). So, the only solution I found to correct this problem was to automatically detect which APN is required.

This might also improve the ease of deployment of your programs. I like putting as much as possible auto-detection / auto-configuration code as possible. It takes a little time to write it but saves a lot of troubles (last minute configuration, human errors, human explanations, etc.).

The APN auto-detection code is like that :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
public static String AutoDetectApn( ATCommand atc ) {
 
    String[] apnList = {
        "\"gprs\",\"objcobytel.com\",\"\",\"\",\"88.191.11.20\",0", // Bouygues Telecom : "Objet communiquant" / "ObjetCo" / "ObjCo"
        "\"gprs\",\"m2minternet\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"internet-entreprise\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"a2bouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"b2bouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"ebouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"movistar.es\",\"movistar\",\"movistar\",\"88.191.11.20\",0",
        "\"gprs\",\"orange\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"orange.fr\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"websfr\",\"\",\"\",\"88.191.11.20\",0"
    };
 
 
   synchronized (atc) {
        System.out.println( "Waiting 30s..." );
        try {
            Thread.sleep( 30000 );
        } catch (InterruptedException ex) {
            ex.printStackTrace();
        }
 
        for (int i = 0; i < apnList.length; ++i) {
            try {
                String apn = apnList[i];
                System.out.println( "Trying apn " + apn + "..." );
                atc.send( "AT^SJNET=" + apn + "\r" );
 
                SocketConnection conn = (SocketConnection) Connector.open( "socket://88.191.11.20:80" );
                conn.close();
                return apn;
            } catch (Exception ex) {
                System.out.println( "Failed : " + ex.getClass() + " : " + ex.getMessage() );
            }
        }
    }
 
 
    // We couldn't find any APN
    return null;
 
}

And it is only launched when the SIM Card has been changed. To detect that, I have some simple code like this :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{ // APN auto-detection
 
    String imsi;
 
    synchronized (atc) {
         imsi = Common.getIMSI( atc );
    }
 
    if (settings.confLastImsi.compareTo( imsi ) != 0) {
        System.out.println( "Sim card changed ! Auto-detecting APN..." );
 
        synchronized (atc) {
            String apn = Common.AutoDetectApn( atc );
 
            if (apn != null)
                reg.confAPN = apn;
        }
 
        settings.confLastImsi = imsi;
 
        // Whatever happens, we still need to save the current IMSI
        settings.Save();
    }
}

Reference :
The forum that helped me solve this problem
Translating Hex to Text

com0com : Create double paired virtual ports

Just a quick note for an interesting project I recently discovered :

The com0com project enables you to create pairs of virtual ports connected with each other. What is so great with that ?
You still have a lot of tools that use com ports to communicate (industrial hardware, old software, gps chips, etc.). It enables you to emulate this software or to make them communicate with “something else” as you wish.

You can build remotly connected serial port or serial hardware emulators (an emulator for a GPS chip for instance).

And the good news is that it works on Windows 2000, Windows XP, Windows Server 2003, Windows Vista, Windows 7 (and their respective 64 bits editions). I know Windows 7 isn’t in the official list. But I tested it with it and it works fine. On the x64 edition of Vista or Windows 7, you have to to use the Driver Signature Enforcement Overrider to install the unsigned com0com driver.

The com0com project.

Lighttpd + Mono ASP.Net : The right configuration

As I already told before, I love the Mono project. It enables to run the powerful Microsoft .Net Framework on UNIX/Linux/BSD systems.

I recently wanted to test a very cool feature of ASP.Net on a mono server. So I did a little

1
apt-get install lighttpd mono-fastcgi-server2 -y

The feature I wanted to try was a web scripting method ( with the “[WebMethod]” attribute) exporting some JSON directly from your method return value.

Here is the web scripting method declaration :

1
2
3
4
5
6
7
8
9
10
 
[ScriptService]
public partial class _Default:Page {
 
	[WebMethod]
	[ScriptMethod( ResponseFormat = ResponseFormat.Json, UseHttpGet = false )]
	public static String TestMethod() {
		return DateTime.Now.ToString();
	}
}

And here is the javascript / JQuery code that gets the data :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
<div id="toto"></div>
 
<script type="text/javascript">
    function doIt() {
        $(document).ready(function() {
            $.ajax({
                type: "POST",
                url: "/Default.aspx/TestMethod",
                contentType: "application/json; charset=utf-8",
                data: "{}",
                dataType: "json",
                success: AjaxSucceeded,
                error: AjaxFailed
            });
        });
        function AjaxSucceeded(result) {
            document.getElementById("toto").innerHTML = result.d;
        }
        function AjaxFailed(result) {
           document.getElementById("toto").innerHTML = "Error : "+result.status + ' ' + result.statusText;
        }
        setTimeout("doIt()", 1000);
    }
    doIt();
 
</script>

In my /etc/lighttpd/conf-enabled/10-fastcgi.conf file, I had this :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
server.modules   += ( "mod_fastcgi" )
fastcgi.server    = (
        "" => ((
                "socket" => mono_shared_dir + "fastcgi-mono-server",
                "bin-path" => mono_fastcgi_server,
                "bin-environment" => (
                        "PATH" => "/bin:/usr/bin:" + mono_dir + "bin",
                        "LD_LIBRARY_PATH" => mono_dir + "lib:",
                        "MONO_SHARED_DIR" => mono_shared_dir,
                        "MONO_FCGI_LOGLEVELS" => "Standard",
                        "MONO_FCGI_LOGFILE" => "/var/log/lighttpd/mono.log",
                        "MONO_FCGI_ROOT" => mono_fcgi_root,
                        "MONO_FCGI_APPLICATIONS" => mono_fcgi_applications
                ),
                "max-procs" => 1,
                "check-local" => "disable"
        ))
)

Everytime I launched a call from javascript, I got (with JS) I got a “405 Method not allowed”. Well, that was pretty disturbing. Mostly because a google search on this didn’t give me anything.
My first thought was that mono didn’t react the same way the .Net framework does. But this isn’t it. It came from my crappy lighttpd config file. It didn’t search for the Default.aspx file but for the Default.aspx/TesMethod.
What you need to do is set :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
fastcgi.server    = (
        ".aspx" => ((
            # Everything you have in your file (like my file above)
        )
 
fastcgi.map-extensions = (
        ".asmx"   => ".aspx",
        ".ashx"   => ".aspx",
        ".asax"   => ".aspx",
        ".ascx"   => ".aspx",
        ".soap"   => ".aspx",
        ".rem"    => ".aspx",
        ".axd"    => ".aspx",
        ".cs"     => ".aspx",
        ".config" => ".aspx",
        ".dll"    => ".aspx"
)

And that will even improve your performances because any other file will be handled directly by lighttpd.

One quick note : There’s something weird with VS2008 WebDev.WebServer.Exe, the WebMethod request takes at least 1.0s to complete. On the same host with XSP, it’s around 15 ms. And on the Celeron 2.6Ghz linux server with lighttpd it’s around 12 ms. And on a IIS 7 server (bundled with Windows 7), it takes 12 ms. So why is the Visual Studio’s WebDev.WebServer so slow ?

By the way, why should we use a Linux + lighttpd + Mono server when we can use a Windows + IIS + ASP.Net server ? The reason is mainly that for the same usage, Linux consumes less resources and it’s easier to customize to suit your needs.

An other question you might have : Why use JQuery + JSON when you can use everything in the Microsoft AJAX Framework ? The main reason is that I really have the feeling to lose control with Microsoft AJAX Framework, I don’t like the huge automatically generated code. And it doesn’t make really fast web interfaces. With JQuery, everything goes faster and it’s way simpler to understand (and debug).

Related :
Using JQuery with ASP.Net

TC65 : Slow UDP reception

Receiving UDP datagrams on the TC65 is easy but inefficient. You need to create a new thread if you want to receive data asynchronously. There’s no way of knowing if a new datagram has been received or not without hanging on the ::receive( Datagram ); method. This is quite weird considering you can do it in TCP.

The real problem is it’s freaking slow. The Cinterion documentation just tells you “This method blocks until a datagram is received.”. What it doesn’t say is that nearly each time, it blocks for 100 to 700 ms after UDP datagrams have actually been received. And worse, this slowness/sleeping avoids the program from treating data and finally throws a little java.io.IOException exception : “No buffer space available”. I tried to give the udp receiving thread a higher priority than the other threads, I tried to make the TCP reception thread sleep a lot just in case it would lock some kind of network object, I looked on how could this method be implemented but couldn’t find a solution and don’t believe there’s one.

By the way, most of the codes on UDP data reception are lame. They all show the same stupid synchronous code. The real code you will need is something like that :

You can remove the Logger class calls. It enables me to make some build-specific (mostly logging) code. The “if” false conditions are remove by the java compiler (which is actually mostly a pre-compiler). I will (quickly this time) talk about this someday…

This program isn’t TC65 specific, it should work with any MIDP enabled J2ME device.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
package Network.UDPReceiver;
 
import Common.Logger;
import java.io.IOException;
import javax.microedition.io.Datagram;
import javax.microedition.io.DatagramConnection;
 
/**
 * UDP event-like receiving class
 * @author Florent Clairambault
 */
public class UDPReceiver implements Runnable {
 
    private IUDPReceive _receivingClass;
    private DatagramConnection _udpConn;
    private Datagram _udpDatagram;
    private Thread _thread;
    private boolean _loop;
 
    /**
     * Creates and launchs the receiver
     * @param conn The connection
     * @param receivingClass The receiving class of the UDP receive event
     */
    public UDPReceiver(DatagramConnection conn, IUDPReceive receivingClass) {
        _udpConn = conn;
        _receivingClass = receivingClass;
        init();
        start();
    }
 
    /**
     * Prepares the datagram used to receive data
     */
    private void init() {
        if (Logger.E_VERBOSE)
            Logger.Log( 32, "UDPReceiver.init();" );
        try {
            int maxLength = _udpConn.getMaximumLength();
            if (Logger.E_DEBUG) {
                Logger.Log( 39, "maxLength=" + maxLength );
            }
            _udpDatagram = _udpConn.newDatagram( maxLength );
        } catch (IOException ex) {
 
            // This should NEVER happen !
            if (Logger.E_CRITICAL)
                Logger.Log( 37, "UDPReceiver.init", ex );
        }
    }
 
    /**
     * The actual data reception
     * @return the data received
     * @throws java.io.IOException When something fais, it means we have to stop
     */
    public synchronized byte[] receiveUdpFrame() throws IOException {
        if (Logger.E_DEBUG)
            Logger.Log( 46, "UDPReceiver.receiveUdpFrame();" );
 
        _udpConn.receive( _udpDatagram );
        int size = _udpDatagram.getLength();
        if (Logger.E_DEBUG)
            Logger.Log( 49, "size = " + size );
 
        // We copy the data so that it can be used on an other thread
        byte[] data = new byte[size];
        System.arraycopy( _udpDatagram.getData(), 0, data, 0, size );
 
        // These two lines might seem weird but are the more efficient way
        // to prepare next datagram reception.
        _udpDatagram.reset();
        _udpDatagram.setLength( _udpDatagram.getData().length );
 
        if (Logger.E_DEBUG)
            Logger.Log( 63, "UDPReceiver.receiveUdpFrame : ok !" );
 
        return data;
    }
 
    /**
     * Starts the receiving thread
     */
    private void start() {
        _loop = true;
 
        _thread = new Thread( this, "udp" );
        _thread.start();
    }
 
    /**
     * Plan to stop
     * 
     * This method is in fact useless. The thread will stop when the
     * DatagramConnection (_udpConn) will be closed.
     */
    public void stop() {
        _loop = false;
    }
 
    /**
     * The thread method
     */
    public void run() {
        try {
            while (_loop) {
                if (Logger.E_DEBUG)
                    Logger.Log( 70, "UDPReceiver : running..." );
 
                byte[] data = receiveUdpFrame();
 
                // We "throw" an event-like method
                _receivingClass.UdpDataReceived( data );
            }
        } catch (IOException ex) {
            if (Logger.E_DEBUG)
                Logger.Log( 54, "UDPReceiver.run", ex );
 
            // The connection must have been closed, we have to stop !
            _loop = false;
        } catch (Exception ex) {
            if (Logger.E_CRITICAL)
                Logger.Log( 119, "UDPReceiver.run", ex );
        }
    }
}

The “mother” class has implements this IUDPReceiver interface :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
package Network.UDPReceiver;
 
/**
 * Interface an asynchronous UDP receiving class must implement
 * @author Florent Clairambault
 */
public interface IUDPReceive {
 
    /**
     * Called when a datagram is received on a connection
     * @param data Content of the datagram
     * 
     * We assume that the receiving class already knows the sender of the
     * datagram because it's a connection established with an "UDP server".
     */
    void UdpDataReceived( byte[] data );
}

The “mother” class should open the connection like that :

1
2
_udpConn = (UDPDatagramConnection) Connector.open( "datagram://" + server );
_udpReceiver = new UDPReceiver( _udpConn, this );

where server is an adresse like that “x.x.x.x:y”. It can also be used as a UDP server with “:y”.

And has to implement the interface IUDPReceiver interface.

Note on receiving UDP over GPRS

Most GPRS connections (every one of them unless you subscribe to special options), are “NATed” (using a NAT router). They are behind a private network (10.x.x.x/8, 192.168.x.x/16, 172.16.x.x/12). That prevents the equipments from being directly accessible (by other means than their phone number).

On TCP, you have to establish a connection with a remote host (that you’re quite likely to call a “server”), and then the server can answer on the link. You have a reliable bi-directional stream you can use to transmit anything you’d like. You can keep the connection open running forever, you just have to send data or set the KeepAlive option. NAT routers break TCP connections when nothing has been transmitted on it for 1 hour to 4 days. This has even been considered like a “bug” by some people because the NAT router reach its memory limit or the max number of TCP connections it can translate (from private to public).
On the TC65, it means you can communicate over a TCP connections for months. You just have to make sure that it’s still alive by providing a little ping mechanism in your protocol (you can send pings every 15/30 minutes for instance).

On UDP, it’s roughly the same thing except there’s no connection. In fact, UDP is minimalist (it only has a 8 bytes header). You have no mechanism for connection establishment, you just throw your data (and by chance, it will be received). The router can’t possibly know when the transmission has ended. So when the router supports bi-directional UDP, it just tries to keep track of the last connections established for a limited time. My personal tomato firmware enabled router, keeps track of the connections when you sent data from LAN to WAN in the last 30 seconds, and that goes to 180 seconds when the other side replies.

So, basically what you need to do to set up a bi-directionnal UDP connection is sending some data from the equipment to the server each 25 seconds. My guess is that even empty UDP packets (8 bytes sized packet) should work.

The most simple/secure way to do this (don’t forget that UDP datagrams can be dropped on their way to the server), is to establish a communication with the server in TCP. Then, ask for a unique identifier (two bytes for instance) and then use it each time you send data on UDP.

If you want to send data between two NATed network, it’s also possible. This little RFC explains it very well. If this seems a little bit to abstract, you should try out wireshark. I always use it when I have a doubts about some low-level network transmission (I like to investigate a lot, just to make sure I’m not missing anything).

I didn’t say it sucks

It didn’t say the TC65 sucks. It’s just not as efficient as I thought it could be. The network stack doesn’t seem to have been built for performances. If you send TCP data or receive UDP data too often, there’s a pretty good chance you will face the same problems. If you found a solution to bypass these problems, please tell me how.

But remember, if you only need to send some data each second (or less often), which is the case in most of the M2M applications, you can be sure the chip will suit your needs.

Cinterion TC65 development tools work on Windows 7

I’ve installed Windows 7 on my PC. I made an upgrade from vista. Everything was saved except the deploy/debug connection to the Cinterion’s SDK. But a little repair operation with the setup installer made it work. It’s pretty cool because when I switched from XP to Vista, well… I switched back to XP due to that problem.

22/12/09 update :
MES doesn’t work fine on any other system than XP. You should try the JObexFTP tool instead.

I know I’m talking a lot about the TC65 these last days, but I’m finishing a little program (on my spare time) on this chip for someone. This little project is interesting because it requires to be fully optimized to match the required goals of performance. It’s pretty new for me, because a second delay time was ok in my last projects. Here, data must be sent at 5Hz (each 200 ms) and if possible at 20Hz (each 50ms), and that’s why I talked about the TC65’s crappy TCP stack.

Don’t forget you can download the SDK here.
I don’t provide the SDK anymore, I had to remove it.