coldtobi's blog

• Main
• Archives
• Albums
• Links
• Admin

• Blog and Website [93]
• DSL RAM [2]
• Electronics and Atmel AVR [11]
• fanctrl [3]
• General [7]
• German [1]
• La Fonera [22]
• Lifetype [17]
• Linux / Debian [33]
• Monetizing A Blog [8]
• my 2 cents [14]
• Netzfunde [18]
• Nice Links [1]
• solarpowerlog [2]
• Thecus N2100 [16]
• Tips and Tricks [18]

• N2M: parallel build dpkg-buildpackage and pdebuild
• Windows CE: Assertion failed in wincore.cpp line 1034
• CakePHP: Baking a Application in Minutes
• Enabling InnoDB support in debian (MySQL)
• Flattr.
• Pollin Net-IO PHP Library
• Link to safe

• RSS 0.90
• RSS 1.0
• RSS 2.0
• Atom

• Privacy Policy
• Disclosure Policy

[Updated] La Fonera: A bunch of Autoupdates (to 0.7.2.2)

On my regular check of my "fon-home-log", I detected, that FON is shipping two updates out since October 23.

I am currently analyzing the changes introduced by these, but until I know more I thinks its safer NOT to apply these updates: On a first sight, they fiddle around with DNS. (Well, dnsmasq to be precise. dnsmasq should not be a problem, but better safe than sorry).

 [UPDATE] 

THE CHANGES I ANALYZED SO FAR ARE DANGEROUS (in terms of losing control)! THEY ARE INSTALLING COOVA (coova.org) AND WITH THIS PACKAGE IT IS POSSIBLE TO LOSE CONTROL OF THE FONERA!  (SSH etc.) 

[UPDATE2]

The package to be installed is only the cova-chillispot. This package does contains not the complete coova firmware, so AFAIK there no possibility to modify the La Fonera using this way.  However, Kolofonium will cease to work, as the code injection using chilli is not longer possible. On the other side, it is still a hotfix: Hard-reseting will still do the job. And for the "downgraded original" firmware its easy with all the known hacks. (Instructions for the hard-reset-part here). When the Foneras are factory-equipped with 0.7.2-2, it is getting tougher but there is also a RS232 and JTAG-header inside of the case. However, either way voids the warranty.

So, now I found some time to get an closer investigation into the update. I decided, as a hotfix is easily undone, I just give it a try. But just applying the update script is boring, and does not give an in-deepth view what they are doing. And also, none wants to lose control, do ya?

DISCLAIMER:  If you break you La Fonera, you own both parts. I won't fix it or buy you a new one. 

Preparing and Analyzing

The first thing to do is a backup of the changes of the file system made so far. This is easy as the device is using a so-called translucent file system: It will only store files that have been altered. As the "changed" part is mounted separately, it is sufficient just to backup the /jffs folder.  So a  brief cd /tmp && tar czf current_jffs.tar.gz /jffs/ bundles the changes in a convenient way. It is done in the /tmp folder, as this will not be stored in /jffs itself, avoiding recursion. To store the tar off site, I scp'ed it to my desktop PC. ( scp <ip_fonera>:/tmp/current_jffs.tar.gz . ) 

Getting the Patch Set. But make sure that it is not executed right away

To get the updates, I used the commands in the instructions supplied by upstream, the script returned by ssh (invoked by the thinclient). As the patchset is not a "plain" tar file, but I modified the script that it only downloads the patch, embraced by the digital signature. Usually I use "dd" to accomplish this, but why not just modify the fonverify-script (located /bin/fonverify) to do the trick? Simply enter an "exit 0" just before the extracted content is executed, which is in the current version around line 158 (see snippet, bold lines added):

busybox tar zx -f $TAR_FILE > /dev/null 2>&1
if [ ! $? -eq 0 ]; then
        echo "ERROR: Decompressing tgz"
        rm $TAR_FILE
        rm -R $UPGRADE_DIRECTORY
        exit 1
fi

#disable proposed execution here!
exit 0

rm $TAR_FILE
if [ -f $UPGRADE_DIRECTORY/upgrade ]; then
        cd $UPGRADE_DIRECTORY
        check_dependencies ./hotfix

After this modifications, I deleted several lines in the downloaded thinclient.sh. The same effect has, but to better illustrate the changes, I simply comment out the  lines to avoid side-effects like rebooting, and added some lines to make sure the updates are placed in individual directories and md5sums to help you validating that you've got the same version of the script. Again, all changes are marked bold.

cd /tmp
wget http://download.fon.com/firmware/update/0.7.1/5/upgrade.fon
md5sum /tmp/upgrade.fon
/bin/fonverify /etc/public_fon_rsa_key.der /tmp/upgrade.fon
#killall chilli
mv /tmp/*upgrade /tmp/upgrade_0715
cd /tmp
wget http://download.fon.com/firmware/update/0.7.2/1/upgrade.fon
md5sum /tmp/upgrade.fon
/bin/fonverify /etc/public_fon_rsa_key.der /tmp/upgrade.fon
#if [ $? -eq 0 ]; then
#       /sbin/reboot
#fi
# killall chilli

mv /tmp/*upgrade /tmp/upgrade_0721
#rm -f /tmp/.thinclient.sh

#exit

After both modifications are done, it is safe to let the script do the work. The output should look like: Please compare the md5sums (bold), just to make sure that you've got the same archive:

root@OpenWrt:~# ./thinclient.sh
Connecting to download.fon.com[213.134.45.191]:80
upgrade.fon          100% |*****************************|   179 KB    00:00 ETA
c7791ad9c9320b6ee986314006609916  /tmp/upgrade.fon
This is a FON hotfix v2 archive
Verified OK
Connecting to download.fon.com[213.134.45.191]:80
upgrade.fon          100% |*****************************|  2120       00:00 ETA
3bfd71e6f995e7a58eb83b64d916bdce  /tmp/upgrade.fon
This is a FON hotfix v2 archive
Verified OK
root@OpenWrt:~#              
 

You'll get two directories in /tmp: update_0715 and update_0722. 

cd /tmp/update_0715; echo "Examining the first update"

Update Procedure 

Now, its time to look into the update. As already mentioned, there is new software on the way. But first lets take a look into the script, that we just stopped being executed: 

root@OpenWrt:~/upgrade_0715# cat upgrade
#!/bin/sh
# $Id$
VERSION=$( cat /etc/fon_version )
REVISION=$( cat /etc/fon_revision )

if [ $VERSION = "0.7.1" ] && [ $REVISION = "5" ];
then
        echo Upgrading...
        grep -v /$ /usr/lib/ipkg/info/dnsmasq.list |xargs rm -f
        grep -v /$ /usr/lib/ipkg/info/chillispot-fon.list | xargs rm -f
        rm /usr/lib/ipkg/info/dnsmasq.list
        rm /usr/lib/ipkg/info/chillispot-fon.list
        # mv /etc/localusers /etc/config/localusers

        tar xzvvf upgrade_0721.tgz -C / 2>&1 >/dev/null

        killall N50chillispot
        killall chilli
        killall watch_chilli
        killall dnsmasq

        ipkg install dnsmasq-fon_2.38-1_mips.ipk
        rm dnsmasq-fon_2.38-1_mips.ipk
        ipkg install coova_4-1_mips.ipk
        rm coova_4-1_mips.ipk

        # Ask for the configuration, just in case
        /etc/init.d/N50chillispot reload

        # Finish upgrade
        cp banner /etc/
        cp fon_version /etc/
        cp fon_revision /etc/

        # Restart services
        lock -u /var/run/restart-services
        exit 0
else
        exit 1
fi

A whole lot to do for the La Fonera. Lets analyze a little bit. First there is the version check, but we won't be here, if  it is not matching: Fon only delivers appropriate hot fixes, considering the reported version. So the first interesting lines are the grep lines. Lets see what they are returning -- before they delete some files which might still be great to have:

root@OpenWrt:~# grep -v /$ /usr/lib/ipkg/info/dnsmasq.list
/etc/dnsmasq.conf
/usr/sbin/dnsmasq
root@OpenWrt:~# grep -v /$ /usr/lib/ipkg/info/chillispot-fon.list
/etc/init.d/N50chillispot
/etc/chilli.conf
/usr/sbin/chilli
/usr/sbin/chilli_radconfig

The chilli executeable and some configuration files, that would be in the way if not deleted manually. This is working around the ipkg-bug (fon, wouldn't it better to fix that sometime?). 

Tar File Contents 

The tar extracts these files to their locations:

/etc/config/whitelist.dnsmasq (new file)
/etc/config/hosts
/sbin/ifup
/www/cgi-bin (empty dir)
/www/status.sh (new file)

With the transition to the new dnsmasq and coova, some handling seems to have changed. Some of them a not yet clear to me, for example the dnsmasq-whitelist. Why is unclear, and one can only guess by the name. But the list in this file can be updated by the chilli-configuration. New entries out of /tmp/chilli.conf will be parsed every time the chilli is started and added to this whitelist. See later for details.

/etc/hosts contains one weird host: It maps several domains to a address in the special block 192.0.2.22: webmail22b.orange.fr, compte.orange.fr, smsmms2.orange.fr, r.orange.fr ,contacts.orange.fr. Why I don't know why they are doing this, this will definitely break things: At least, these domains will unreachable using the La Fonera. 

/www/status.sh seems to me to be an enhancement, a nice looking status page.  

/sbin/ifup just adds one line: iwpriv ath0 ap_bridge 0, which is good as it separated to connected clients. 

(BTW, fon ships all these files twice: One time they are already in the update.fon-archive and a second time in the upgrade_0721.tgz, which is contained in the upgrade.fon. All the files are identical. But only the files in the tgz are used.)

IPKGing and other things.

Let's continue in the upgrade-script. The killalls just stops all services related to the chillispot and of course dnsmasq, which will be installed with the following  ipkg lines. Lets take a closer look at these packages:

dnsmasq is gonna to be updated from version 2.27 to version 2.38-fon. Why has the version appended "-fon". Has Fon modified dnsmasq? I think so. Unfortunatly one cannot tell, which parts has been modified, as I was unable to find the updated source code. (I'm going to  look into the La Fonera+-Source, maybe I can find some hints there.) The hints I found let me suppose that it was modified: Doing a strings on the executeable reveals some strings not present in current dnsmasqs (debian, 2.40): For example, the "/etc/whitelist.dnsmasq" installed before, is used refrenenced by the new version. 

Coova used is a replacement for the chillispot. There would be a complete firmware suit, but this is not be used. The version is "chillispot 1.0-coova.4". Currently I cannot tell, to much about the improvements compared to chilli, I have to dig into the documentation; but on the first impression, coova is really souperior to chilli. My plans are, that I will test coova in a chroot-jail. Fortunatly, La Fonera's busybox does have chroot.

Killing kolofonium

With the coova packages, s new version of the N50chillispot is installed. This file has some interesting changes, so it is worth to take a deeper look into it. Lets take a look at the first snippet:

    
DEVICE=$(cat /etc/fon_device)
VERSION=$(cat /etc/fon_version)
REVISION=$(cat /etc/fon_revision)
(...)
RADIUSADMUSR=FON01-${DEVICE}-${VERSION}.${REVISION}

With this new way of identification, The Fonera will tell the radius server, that it is really a fonera, and exactly which version of the firmware is running inside. Before, the value was "chillispot".

On to the next snippet, which is completely new code:

TMP_C=/tmp/chilli.conf
 
WHITELIST=/etc/config/whitelist.dnsmasq

(...)

parse_whitelist() {
        if [ ! -z $WHITELIST ]; then
                awk '/newdomain/ { print $2 }' $TMP_C |
                awk -F. '/[a-zA-Z0-9-_]/ { print }' |
                sed s/,/n/g > $WHITELIST
        # Remove IPs that may have skipped the validation
        cat $WHITELIST | grep -v [0-9]$ > $WHITELIST

                sed /^newdomain/d -i $TMP_C
        fi
}

The whitelist -- as the name suggest -- has to do with the dnsmasq used, but details are unclear as the trace is lost somewhere inside its binary. What you can tell from the snippet above is, that this whitelist can be modified using the chilli-configuration-file. That file is assembled through by coova/chillispot with informations from the fon servers. And if there are special crafted lines in there they will be added to the whitelist. The name "newdomain" suggest, that they want to whitelist domain names. Afterwards, they make sure that no IP-Addresses are whitelistet by eliminating all lines ending with numbers. (BTW: They either were in luck or knew that OpenWrt's ping cannot ping hexadecimal ips. Try  "ping 0x7F.0x00.0x00.0x01" on a linux machine and on the Fonera to see the difference.)  

The next snipped:

 radconfig() {
    /usr/sbin/chilli_radconfig
        -c /dev/null
        --radiusserver1="$RADIUSSERVER"
        --radiussecret="$RADIUSSECRET"
        --adminuser="$RADIUSADMUSR"
        --adminpasswd="$RADIUSADMPWD"
        --radiusnasid="$MAC"
        --dhcpif $wifi_ifname
        --wwwbin=/bin/true
        --ipup=/bin/true
        --ipdown=/bin/true
        > $TMP_C
    
    parse_whitelist

The three lines mapping commands to /bin/true are the ones defeating kolofonium, and is beside that a long awaited security fix: There were some possible scenarios, where malicious people abuse this and possibly could do bad things with the box. (eg. with DNS-spoofing ).

After the rad-server-configuration is fetched using chilli_radconfig, the above mentioned whitelisting is performed.

Some lines later, in the do_start(), they almost repeat the lines (mapping two more commands to /bin/true). But this time, DNS settings have changed: Here they throw in the dnsmasq-settings hardcoded to 192.168.182.1:

     /usr/sbin/chilli
        --dns1="192.168.182.1"
        --dns2="192.168.182.1"
        --radiusnasid="$MAC"
        --dhcpif $wifi_ifname
        --papalwaysok
        --pidfile=$PID_F
        --localusers=/etc/config/localusers
        --wwwbin=/bin/true
        --ipup=/bin/true
        --ipdown=/bin/true
        --conup=/bin/true
        --condown=/bin/true  

All other changes made to this  file are of minor nature: The "check-if-chilli-has-died" function has been moved into a seperate file, and the script now nows to "reload", which basically re-retrieves a new chilli-configuration from the radius server. However, they quite rearranged the script and made it more modular. But it is still coded quite ugly throuhout the file.

Other files

The other files just make sure, that the infrastructure and version numbers are updated. The same business  as usual.

Upgrade to 0.7.2-2 

This upgrade contains only one file: /bin/thinclient. I think they just forgot to inlcude it into the other packages. I have the impression, that inclusion of this file is only a ritual, maybe they just want try to reenable the "autoupdates". The only change made to the file -- compared to the older version -- is, that the downloaded script is now getting a parameter. The parameter just tells the script, if it was called  due a system startup or because of a cron-job. Thats all. 

Just for completness, here's the line in question: 

. /tmp/.thinclient.sh  $1
 

Conclusion

Currently, I cannot tell if the new pacakges will jepodize control over the box. As already mentioned, this needs more tests. But this will be a topic of another article.

However, I really begin to love the idea to place all the FON-stuff into its own chroot.