One weird bug on the XT75

You might have faced this error with your X765 chip. The program crashes with this error :

1
2
3
^EXIT 00010000,02d6414253485f4c434c2c70726f6365647572655265636f72645f702d3e6e756d6265724f66526567697374657265645461736b73203c3d204d41585f4e4f5f524547495354455245445f5441534b53
 
^SHUTDOWN

If you convert the hex array to some text, that will give you :

1
#ABSH_LCL,procedureRecord_p->numberOfRegisteredTasks <= MAX_NO_REGISTERED_TASKS

Which might mean something to someone. But the point is, you won’t find any help. Mostly because it’s an uncommon error.

The error comes from the GPRS connection management. I had it when I was using a wrong APN (with the “AT^SJNET” command) to connect to a host. In my program, connection failed with a classic IOException (“Profile not found”) but 1 or 2 minutes later, the chip was ALWAYS crashed (with the “^EXIT” URC). So, the only solution I found to correct this problem was to automatically detect which APN is required.

This might also improve the ease of deployment of your programs. I like putting as much as possible auto-detection / auto-configuration code as possible. It takes a little time to write it but saves a lot of troubles (last minute configuration, human errors, human explanations, etc.).

The APN auto-detection code is like that :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
public static String AutoDetectApn( ATCommand atc ) {
 
    String[] apnList = {
        "\"gprs\",\"objcobytel.com\",\"\",\"\",\"88.191.11.20\",0", // Bouygues Telecom : "Objet communiquant" / "ObjetCo" / "ObjCo"
        "\"gprs\",\"m2minternet\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"internet-entreprise\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"a2bouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"b2bouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"ebouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"movistar.es\",\"movistar\",\"movistar\",\"88.191.11.20\",0",
        "\"gprs\",\"orange\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"orange.fr\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"websfr\",\"\",\"\",\"88.191.11.20\",0"
    };
 
 
   synchronized (atc) {
        System.out.println( "Waiting 30s..." );
        try {
            Thread.sleep( 30000 );
        } catch (InterruptedException ex) {
            ex.printStackTrace();
        }
 
        for (int i = 0; i < apnList.length; ++i) {
            try {
                String apn = apnList[i];
                System.out.println( "Trying apn " + apn + "..." );
                atc.send( "AT^SJNET=" + apn + "\r" );
 
                SocketConnection conn = (SocketConnection) Connector.open( "socket://88.191.11.20:80" );
                conn.close();
                return apn;
            } catch (Exception ex) {
                System.out.println( "Failed : " + ex.getClass() + " : " + ex.getMessage() );
            }
        }
    }
 
 
    // We couldn't find any APN
    return null;
 
}

And it is only launched when the SIM Card has been changed. To detect that, I have some simple code like this :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{ // APN auto-detection
 
    String imsi;
 
    synchronized (atc) {
         imsi = Common.getIMSI( atc );
    }
 
    if (settings.confLastImsi.compareTo( imsi ) != 0) {
        System.out.println( "Sim card changed ! Auto-detecting APN..." );
 
        synchronized (atc) {
            String apn = Common.AutoDetectApn( atc );
 
            if (apn != null)
                reg.confAPN = apn;
        }
 
        settings.confLastImsi = imsi;
 
        // Whatever happens, we still need to save the current IMSI
        settings.Save();
    }
}

Reference :
- The forum that helped me solve this problem
- Translating Hex to Text

com0com : Create double paired virtual ports

Just a quick note for an interesting project I recently discovered :

The com0com project enables you to create pairs of virtual ports connected with each other. What is so great with that ?
You still have a lot of tools that use com ports to communicate (industrial hardware, old software, gps chips, etc.). It enables you to emulate this software or to make them communicate with “something else” as you wish.

You can build remotly connected serial port or serial hardware emulators (an emulator for a GPS chip for instance).

And the good news is that it works on Windows 2000, Windows XP, Windows Server 2003, Windows Vista, Windows 7 (and their respective 64 bits editions). I know Windows 7 isn’t in the official list. But I tested it with it and it works fine. On the x64 edition of Vista or Windows 7, you have to to use the Driver Signature Enforcement Overrider to install the unsigned com0com driver.

- The com0com project.

Posted in English. Tags: , . No Comments »

Lighttpd + Mono ASP.Net : The right configuration

As I already told before, I love the Mono project. It enables to run the powerful Microsoft .Net Framework on UNIX/Linux/BSD systems.

I recently wanted to test a very cool feature of ASP.Net on a mono server. So I did a little

1
apt-get install lighttpd mono-fastcgi-server2 -y

The feature I wanted to try was a web scripting method ( with the “[WebMethod]” attribute) exporting some JSON directly from your method return value.

Here is the web scripting method declaration :

1
2
3
4
5
[WebMethod]
[ScriptMethod( ResponseFormat = ResponseFormat.Json, UseHttpGet = false )]
public static String TestMethod() {
	return DateTime.Now.ToString();
}

And here is the javascript / JQuery code that gets the data :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
<div id="toto"></div>
 
<script type="text/javascript">
    function doIt() {
        $(document).ready(function() {
            $.ajax({
                type: "POST",
                url: "/Default.aspx/TestMethod",
                contentType: "application/json; charset=utf-8",
                data: "{}",
                dataType: "json",
                success: AjaxSucceeded,
                error: AjaxFailed
            });
        });
        function AjaxSucceeded(result) {
            document.getElementById("toto").innerHTML = result.d;
        }
        function AjaxFailed(result) {
           document.getElementById("toto").innerHTML = "Error : "+result.status + ' ' + result.statusText;
        }
        setTimeout("doIt()", 1000);
    }
    doIt();
 
</script>

In my /etc/lighttpd/conf-enabled/10-fastcgi.conf file, I had this :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
server.modules   += ( "mod_fastcgi" )
fastcgi.server    = (
        "" => ((
                "socket" => mono_shared_dir + "fastcgi-mono-server",
                "bin-path" => mono_fastcgi_server,
                "bin-environment" => (
                        "PATH" => "/bin:/usr/bin:" + mono_dir + "bin",
                        "LD_LIBRARY_PATH" => mono_dir + "lib:",
                        "MONO_SHARED_DIR" => mono_shared_dir,
                        "MONO_FCGI_LOGLEVELS" => "Standard",
                        "MONO_FCGI_LOGFILE" => "/var/log/lighttpd/mono.log",
                        "MONO_FCGI_ROOT" => mono_fcgi_root,
                        "MONO_FCGI_APPLICATIONS" => mono_fcgi_applications
                ),
                "max-procs" => 1,
                "check-local" => "disable"
        ))
)

Everytime I launched a call from javascript, I got (with JS) I got a “405 Method not allowed”. Well, that was pretty disturbing. Mostly because a google search on this didn’t give me anything.
My first thought was that mono didn’t react the same way the .Net framework does. But this isn’t it. It came from my crappy lighttpd config file. It didn’t search for the Default.aspx file but for the Default.aspx/TesMethod.
What you need to do is set :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
fastcgi.server    = (
        ".aspx" => ((
            # Everything you have in your file (like my file above)
        )
 
fastcgi.map-extensions = (
        ".asmx"   => ".aspx",
        ".ashx"   => ".aspx",
        ".asax"   => ".aspx",
        ".ascx"   => ".aspx",
        ".soap"   => ".aspx",
        ".rem"    => ".aspx",
        ".axd"    => ".aspx",
        ".cs"     => ".aspx",
        ".config" => ".aspx",
        ".dll"    => ".aspx"
)

And that will even improve your performances because any other file will be handled directly by lighttpd.

One quick note : There’s something weird with VS2008 WebDev.WebServer.Exe, the WebMethod request takes at least 1.0s to complete. On the same host with XSP, it’s around 15 ms. And on the Celeron 2.6Ghz linux server with lighttpd it’s around 12 ms. And on a IIS 7 server (bundled with Windows 7), it takes 12 ms. So why is the Visual Studio’s WebDev.WebServer so slow ?

By the way, why should we use a Linux + lighttpd + Mono server when we can use a Windows + IIS + ASP.Net server ? The reason is mainly that for the same usage, Linux consumes less resources and it’s easier to customize to suit your needs.

An other question you might have : Why use JQuery + JSON when you can use everything in the Microsoft AJAX Framework ? The main reason is that I really have the feeling to lose control with Microsoft AJAX Framework, I don’t like the huge automatically generated code. And it doesn’t make really fast web interfaces. With JQuery, everything goes faster and it’s way more simpler to understand (and debug).

Related :
- Using JQuery with ASP.Net

TC65 : Slow UDP reception

Receiving UDP datagrams on the TC65 is easy but inefficient. You need to create a new thread if you want to receive data asynchronously. There’s no way of knowing if a new datagram has been received or not without hanging on the ::receive( Datagram ); method. This is quite weird considering you can do it in TCP.

The real problem is it’s freaking slow. The Cinterion documentation just tells you “This method blocks until a datagram is received.”. What it doesn’t say is that nearly each time, it blocks for 100 to 700 ms after UDP datagrams have actually been received. And worse, this slowness/sleeping avoids the program from treating data and finally throws a little java.io.IOException exception : “No buffer space available”. I tried to give the udp receiving thread a higher priority than the other threads, I tried to make the TCP reception thread sleep a lot just in case it would lock some kind of network object, I looked on how could this method be implemented but couldn’t find a solution and don’t believe there’s one.

By the way, most of the codes on UDP data reception are lame. They all show the same stupid synchronous code. The real code you will need is something like that :

You can remove the Logger class calls. It enables me to make some build-specific (mostly logging) code. The “if” false conditions are remove by the java compiler (which is actually mostly a pre-compiler). I will (quickly this time) talk about this someday…

This program isn’t TC65 specific, it should work with any MIDP enabled J2ME device.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
package Network.UDPReceiver;
 
import Common.Logger;
import java.io.IOException;
import javax.microedition.io.Datagram;
import javax.microedition.io.DatagramConnection;
 
/**
 * UDP event-like receiving class
 * @author Florent Clairambault
 */
public class UDPReceiver implements Runnable {
 
    private IUDPReceive _receivingClass;
    private DatagramConnection _udpConn;
    private Datagram _udpDatagram;
    private Thread _thread;
    private boolean _loop;
 
    /**
     * Creates and launchs the receiver
     * @param conn The connection
     * @param receivingClass The receiving class of the UDP receive event
     */
    public UDPReceiver(DatagramConnection conn, IUDPReceive receivingClass) {
        _udpConn = conn;
        _receivingClass = receivingClass;
        init();
        start();
    }
 
    /**
     * Prepares the datagram used to receive data
     */
    private void init() {
        if (Logger.E_VERBOSE)
            Logger.Log( 32, "UDPReceiver.init();" );
        try {
            int maxLength = _udpConn.getMaximumLength();
            if (Logger.E_DEBUG) {
                Logger.Log( 39, "maxLength=" + maxLength );
            }
            _udpDatagram = _udpConn.newDatagram( maxLength );
        } catch (IOException ex) {
 
            // This should NEVER happen !
            if (Logger.E_CRITICAL)
                Logger.Log( 37, "UDPReceiver.init", ex );
        }
    }
 
    /**
     * The actual data reception
     * @return the data received
     * @throws java.io.IOException When something fais, it means we have to stop
     */
    public synchronized byte[] receiveUdpFrame() throws IOException {
        if (Logger.E_DEBUG)
            Logger.Log( 46, "UDPReceiver.receiveUdpFrame();" );
 
        _udpConn.receive( _udpDatagram );
        int size = _udpDatagram.getLength();
        if (Logger.E_DEBUG)
            Logger.Log( 49, "size = " + size );
 
        // We copy the data so that it can be used on an other thread
        byte[] data = new byte[size];
        System.arraycopy( _udpDatagram.getData(), 0, data, 0, size );
 
        // These two lines might seem weird but are the more efficient way
        // to prepare next datagram reception.
        _udpDatagram.reset();
        _udpDatagram.setLength( _udpDatagram.getData().length );
 
        if (Logger.E_DEBUG)
            Logger.Log( 63, "UDPReceiver.receiveUdpFrame : ok !" );
 
        return data;
    }
 
    /**
     * Starts the receiving thread
     */
    private void start() {
        _loop = true;
 
        _thread = new Thread( this, "udp" );
        _thread.start();
    }
 
    /**
     * Plan to stop
     * 
     * This method is in fact useless. The thread will stop when the
     * DatagramConnection (_udpConn) will be closed.
     */
    public void stop() {
        _loop = false;
    }
 
    /**
     * The thread method
     */
    public void run() {
        try {
            while (_loop) {
                if (Logger.E_DEBUG)
                    Logger.Log( 70, "UDPReceiver : running..." );
 
                byte[] data = receiveUdpFrame();
 
                // We "throw" an event-like method
                _receivingClass.UdpDataReceived( data );
            }
        } catch (IOException ex) {
            if (Logger.E_DEBUG)
                Logger.Log( 54, "UDPReceiver.run", ex );
 
            // The connection must have been closed, we have to stop !
            _loop = false;
        } catch (Exception ex) {
            if (Logger.E_CRITICAL)
                Logger.Log( 119, "UDPReceiver.run", ex );
        }
    }
}

The “mother” class has implements this IUDPReceiver interface :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
package Network.UDPReceiver;
 
/**
 * Interface an asynchronous UDP receiving class must implement
 * @author Florent Clairambault
 */
public interface IUDPReceive {
 
    /**
     * Called when a datagram is received on a connection
     * @param data Content of the datagram
     * 
     * We assume that the receiving class already knows the sender of the
     * datagram because it's a connection established with an "UDP server".
     */
    void UdpDataReceived( byte[] data );
}

The “mother” class should open the connection like that :

1
2
_udpConn = (UDPDatagramConnection) Connector.open( "datagram://" + server );
_udpReceiver = new UDPReceiver( _udpConn, this );

where server is an adresse like that “x.x.x.x:y”. It can also be used as a UDP server with “:y”.

And has to implement the interface IUDPReceiver interface.

Note on receiving UDP over GPRS

Most GPRS connections (every one of them unless you subscribe to special options), are “NATed” (using a NAT router). They are behind a private network (10.x.x.x/8, 192.168.x.x/16, 172.16.x.x/12). That prevents the equipments from being directly accessible (by other means than their phone number).

On TCP, you have to establish a connection with a remote host (that you’re quite likely to call a “server”), and then the server can answer on the link. You have a reliable bi-directional stream you can use to transmit anything you’d like. You can keep the connection open running forever, you just have to send data or set the KeepAlive option. NAT routers break TCP connections when nothing has been transmitted on it for 1 hour to 4 days. This has even been considered like a “bug” by some people because the NAT router reach its memory limit or the max number of TCP connections it can translate (from private to public).
On the TC65, it means you can communicate over a TCP connections for months. You just have to make sure that it’s still alive by providing a little ping mechanism in your protocol (you can send pings every 15/30 minutes for instance).

On UDP, it’s roughly the same thing except there’s no connection. In fact, UDP is minimalist (it only has a 8 bytes header). You have no mechanism for connection establishment, you just throw your data (and by chance, it will be received). The router can’t possibly know when the transmission has ended. So when the router supports bi-directional UDP, it just tries to keep track of the last connections established for a limited time. My personal tomato firmware enabled router, keeps track of the connections when you sent data from LAN to WAN in the last 30 seconds, and that goes to 180 seconds when the other side replies.

So, basically what you need to do to set up a bi-directionnal UDP connection is sending some data from the equipment to the server each 25 seconds. My guess is that even empty UDP packets (8 bytes sized packet) should work.

The most simple/secure way to do this (don’t forget that UDP datagrams can be dropped on their way to the server), is to establish a communication with the server in TCP. Then, ask for a unique identifier (two bytes for instance) and then use it each time you send data on UDP.

If you want to send data between two NATed network, it’s also possible. This little RFC explains it very well. If this seems a little bit to abstract, you should try out wireshark. I always use it when I have a doubts about some low-level network transmission (I like to investigate a lot, just to make sure I’m not missing anything).

I didn’t say it sucks

It didn’t say the TC65 sucks. It’s just not as efficient as I thought it could be. The network stack doesn’t seem to have been built for performances. If you send TCP data or receive UDP data too often, there’s a pretty good chance you will face the same problems. If you found a solution to bypass these problems, please tell me how.

But remember, if you only need to send some data each second (or less often), which is the case in most of the M2M applications, you can be sure the chip will suit your needs.

Posted in English. Tags: , , , , . 3 Comments »

Cinterion TC65 development tools work on Windows 7

I’ve installed Windows 7 on my PC. I made an upgrade from vista. Everything was saved except the deploy/debug connection to the Cinterion’s SDK. But a little repair operation with the setup installer made it work. It’s pretty cool because when I switched from XP to Vista, well… I switched back to XP due to that problem.

22/12/09 update :
MES doesn’t work fine on any other system than XP. You should try the JObexFTP tool instead.

I know I’m talking a lot about the TC65 these last days, but I’m finishing a little program (on my spare time) on this chip for someone. This little project is interesting because it requires to be fully optimized to match the required goals of performance. It’s pretty new for me, because a second delay time was ok in my last projects. Here, data must be sent at 5Hz (each 200 ms) and if possible at 20Hz (each 50ms), and that’s why I talked about the TC65’s crappy TCP stack.

Don’t forget you can download the SDK here.

Posted in English. Tags: , . 2 Comments »

Diagnose OTAP problems

I was looking at my analytics stats (which are not so great, but I don’t really care) and I saw that some of you were accessing this blog by searching for “at^sjotap error codes”. This is quite weird.

The only thing you need to test your Over The Air Provisioning (OTAP) is the command :

1
AT^SCFG="Trace/Syslog/OTAP","1"

But you have to make sure you type it right after you typed the “AT^SJOTAP” command. So, if you do it in a terminal. You should copy the syslog enable command, write the “AT^SJOTAP” command and immediatly past the syslog enable command.

What you should have is something like that :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
--> AT^SJOTAP
<-- OK
--> AT^SCFG="Trace/Syslog/OTAP","1"
<-- SYSLOG ENABLED
<--
<-- [OTAP] GPRS connection established.
<-- [OTAP] Try to get http://www.yourwebsite.com/software.jad ...
<-- [OTAP] Connected.
<-- [OTAP] Transfer finished.
<-- [OTAP] Try to get http://www.yourwebsite.com/software.jar ...
<-- [OTAP] Connected.
<-- [OTAP] Transfer finished.
<-- [OTAP] JAM status: 900 Success.
<-- [OTAP] Reboot now.
<-- ^SYSSTART

For those who still want the answer to the “AT^SJOTAP error codes”, you can find it in “Java_UserGuide.pdf” file provided by Cinterion TC65 SDK, page 61 :

  • 900 Success
  • 901 Insufficient memory in filesystem
  • 902 – not supported-
  • 903 – not supported-
  • 904 JAR size mismatch, given size in JAD file does not match real size of jar file
  • 905 Attribute mismatch, one of the mandatory attributes MIDlet-name, MIDlet-version, MIDlet-Vendor in the JAD file does not match those given in the JAR manifest
  • 906 Invalid descriptor, something is wrong with the format of the .jad file
  • 907 invalid JAR, the JAR file was not available under MIDlet-Jar-URL, files could not be extracted from JAR archive, or something else is wrong with the format of the file.
  • 908 incompatible configuration or profile
  • 909 application authentication failure, signature did not match certificate
  • 910 application authorization failure, tried to replace signed with unsigned version
  • 911 -not supported-
  • 912 Delete Notification

The most common error you can make is to have a wrong JAD file, this what it should look like :
The most common error you could make is to do not specify the complete path of your jar file in your jad file. You jad file should look like that :

1
2
3
4
5
6
7
8
MIDlet-1: Midlet, ,MySoft.Midlet
MIDlet-Jar-Size: 29180
MIDlet-Jar-URL: http://87.106.206.30/TC65/mysoft/mysoft.jar
MIDlet-Name: MySoft
MIDlet-Vendor: Cinterion
MIDlet-Version: 1.0
MicroEdition-Configuration: CLDC-1.1
MicroEdition-Profile: IMP-NG

If you still have problems with OTAP, you can leave me a comment.

Posted in English. Tags: , , . 4 Comments »