moiji-mobile

Another period of not blogging. A lot of things have happened. I attented the KDE conference in Dublin, was at OEDEM, went to the awesome Qt DevDays, visited a couple of other facilities and the term just started.
I’m mentally back to KDE development and my personal goals for KDE4 are better QA, smaller memory footprint, faster apps, less bloat, more usable enduser applications and sane error reporting. I still have nightmares understanding why GPG, KMail and Kubuntu won’t work together. “Error initializing the backend” is IMHO not a usable error message and these things need to be changed. I have started writing my first web application using the Django Framework it is called Bonsai and allows you to query SCMs. E.g. the primary usecase is to find deleted files as most modern systems don’t have something like an Attic we are all used to from CVS.

Back to the Freedom and Qtopia Greenphone. It is an incredible device and if you buy one from Trolltech as Free Software developer you even get the sourcecode of the applications. And who needs the source for the kernel, libc, Qtopia libraries and the toolchain itself anyway. I think sourcecode is overrated anyway. Well I’m just kidding.

I took a quick look at the Qtopia Release Notes and I just recognized that only nine more days are missing and Qtopia hasn’t had a GPL release for another year. From 17 documented releases 7 have a GPL release. And the last six releases are without any GPL release at all. From my point of view this makes Qtopia a proprietary platform like Symbian or Windows Mobile. I seriously wish Trolltech commercial success with their platform but from a Free Software point of view the Qtopia platform is just as interesting as Windows Mobile. And Windows Mobile has sadly one advantage, you can actually buy devices running Windows Mobile in Europe. And with handhelds.org we have a project specialised in liberating these devices which secures our freedom. So lazyweb do me a favor and stop believing Qtopia is Free Software.

I find it amusing guyhlem asked Opie to die and do you know what? Opie is dead, unmaintained, stuck with Qt2. So as Opie is dead all people use Qtopia now, right? Sadly this is wrong. The short answer is people that care about freedom, or a maintained platform just use GPE an Gtk+/X11 based environment. The longer answer is when Opie was actively developed the developers used the applications, integrated Opie to new devices and as their were users they knew about the issues and shortcomings. So a lot of real life issues and shortcomings were found and the important ones were hopefully addressed. This is the classic Quality Assurance of Free Software Projects. Code gets released, people test it and find all kind of issues which you are trying to fix. The result is a better integrated, more stable system. I have tried a lot of times to merge Qtopia improvements back into Opie and either I’m completely stupid or most of the stuff that was advertised in the Release Notes didn’t work properly. My favorite example is the Config Cache in the Qtopia Config class. Config is a small Map which maps keys to values using QMap. The keys belong to a specific group. For the current group they have an iterator on this map. Now on write they destroy the QMap and invalidate the iterator. So if you now read or write through this iterator your application will just segfault. So for Opie I have merged and later patched their class to not crash, send patches to Trolltech and do you know what? Config::write in Qtopia will still leave a QMap::Iterator around which is invalid. The other interesting issue was with the fifteen game. They enchanched the graphics of this game and I decdied to merge it. Merging was again straight forward but the application didn’t look right on 640×480. Hey one of the engineers mixed up width with height and we all know these things can happen. So I have written a patch against Qtopia sent it to the opie-forum@trolltech.com and the feedback again was none. I don’t know if they have fixed this issue, fixed it independly or whatever. So the issue with Qtopia until now was the engineers at Trolltech probably don’t have the time to listen to us, to the needs of the community or even accept patches.
Imagine we would have used Qtopia2.1.0 when it was released and ditched Opie. Imagine that we had the resources to make it work well enough on the available platforms (mainly iPAQs, Zaurus). We would have to maintain a patch set for all the bugs we find, no patches from us would be applied. For futher releases we just have to hope that they will do the right thing(tm) and without talking to us still doing what we need. So Qtopia2.1.1 gets releases and we don’t know what bugs were fixed as no ChangeLog will be made available. So we would have to merge all our patches to Qtopia2.1.1 again and maybe even have new patches. And trust me merging 100+ patches to new releases is no joy. And you will do the same thing for Qtopia2.2, Qtopia2.3, Qtopia3.0, Qtopia4.0…. And you sincerly hope that they will do the right thing and fix the bugs you have fixed but reaility is different.We would end up with an ever growing patch set as we find new issues, start polishing applications and do you know what? Trolltech will do the same. So at your next merge point you can either throw your improvements away or try to merge them again. So at one point you might recognize that they won’t do the right thing(tm) when they don’t communicate and there is no way forcing them to.

So what would you do then? I have learned my lessons during Opie development so I’m thankful for this experience and I’m better educated than ever. Probably the most import lesson is “Don’t play with a team when the team doesn’t want to play with you”. So what does that mean? We have always advertised Opie as Qtopia compatible. And as Opie was Free Software, many people thought and still think Qtopia is Free Software as well and we have taken the public preassure away from Trolltech. So we have advertized their platform and created public awarenes even if they didn’t want to play with us. We went to fairs and demonstrated the Free Qtopia, talked with many people from different companies and projects and some of those evolved into paying gigs for Trolltech. And Trolltech went as far as advertising thousands of available Free Software for their platform.

R.I.P Opie, it was a pleasure to hack on you. But there is still hope. There is still the hope that Trolltech will release Qtopia 4.2 GPL and if they somehow promises to regulary release Qtopia sources we might see a Community around Qtopia again.

And Qt4.2 is full of incredible stuff. New feature likes Cascading Style Sheets for Widgets/Applications and the QGraphicsView are just awesome. E.g. the painting engine in Qt can probably be compared to cairo but there is a huge difference. Qt has a fixed point implementation for this engine since day one. So Qtopia4.2 might just be right for your device so let us hope Trolltech will recognize the value of the community (again).

Trolltech announced the GREENPHONE some where last month. It is intended to be a developer device but the design and hardware features are modeled after the average low cost consumer phone. While we have seen on linuxdevices.com articles stating Qt2 is not capable of running on such phones and requiring a higher Bill of Material (BOM), e.g. needing a faster CPU, more flash and more RAM to run. So with the GREENPHONE Trolltech probably tries to demonstrate that it can run on such phones and that it runs well. I think this is a great idea and I hope this works out.

But sadly there are issues with this approach. This phone seems to be locked back down in the past. It runs an ancient and unmaintained major version of the kernel. It is likely to use a gcc that didn’t know that the used processor exists and created slow code for this platform, it is likely to use a zlib, png and jpeg library that has known security issues. I have seen this pattern in the past and I think I know the answer. The award winning Qtopia environment relies on the award winning set of tools from MontaVista. I think this is the corporate ride the dead horse game normally called Return of Investment.

Now to the solution. Trolltech tries to attract OpenSource developers. While I do Free Software I still think they want to attract me. So if they license Qtopia Core and Qtopia Phone under a Free Software compatible license we could create an alternative image that is both smaller and faster than the default installation and make Trolltech use that community but fully API compatible image. It would be a great win for 3rd party developers, it would be great for Trolltech it would be great just for everyone.

So Trolltech please use a suitable license allowing the magic to happen or take the shortcut and hire me and let me work on switching your image over to EABI, switching Qtopia over to uclibc, hacking gcc to produce less code and making the linux footprint smaller. It is up to you.

dvbsnoop is a tool like the known tcpdump but for Digital Video Broadcast. It almost knows about each and every section on AIR and can dump them. It also knows about the the Multiprotocol Encapsulation (MPE) which can contain IP Datagrams.
What would be nice is to use wireshark or tcpdump on these datagrams to analyse them further or to make use of these datagrams in some applications. Now how to get the IP Datagrams from dvbsnoop into the Linux IP stack? The short answer is using tuntap to get the data into the stack. You have the options to use TUN or TAP the difference is that with TAP you write complete Ethernet frames where TUN only operates on IP datagrams. I have decided to use TUN and followed the example to create a tun device and you end up with a filedescriptor and you only need to write the IP Datagrams. Well actually I have wondered about the frame format and have used the IFF_NO_PI option. Within a couple of minutines I was able to do tcpdump -i tun1 and saw UDP packages. Now one can use dvbnet and dvbsnoop to receive IP datagrams over DVB. I prefer dvbsnoop as if this crashes my kernel still works fine in contrast to using dvbnet.
I will post patches of this little hack soon, meanwhile I search for DVB-T receivers capable of receiving the full Transportstream (PID 0x2000)

Before having interned at ROAD I was a heavy user of XEmacs, having used the various elisp scripts from KDE and linux CodingStyle and various other elisp scripts. My .xemacs dir was quite nice and with using gnuserv the startup speed of XEmacs was no issue as well. I’m not good at remembering keyboard shortcuts so XEmacs was pretty much the only choiche I had. Somehow I switched to VIM and were able to remember most of the shortcuts and the easy searching within files, which I was never able to do within XEmacs…

Today was the day, I was used to Gtk+ apps randomly crashing on XErrors on my forwarded X sessions, sadly GNU emacs and XEmacs crashed as well and decided to google and found the -Y option for ssh and now Gtk+/GVim, XEmacs and GNU emacs do not abort on XErrors anymore. Sadly the keymap of GVim is totally wrong and I can not even do curly braces or pipe symbols, so I installed xemacs21 on my system but the fonts still looked ugly and I decided to give GNU emacs a try. Now I have a nicely looking editor with syntax highlighting and lisp under the hood. I think I will give GNU emacs a trial and try to remember the various options and collect elisp files again.

I’m really looking forward to see the release of the GREENPHONE and I hope it comes at an affordable price. One issue that worries me is that it is shipping with a 2.4.19 kernel. I sincerly hope that they have patched all security issues in their ancient kernel, use a gcc 2.95 that actually knows about XScale and can produce good enough code and that the root filesystem is not as ancient as their kernel. I think I will lose all my hairs if they ship a phone with a vulnerable zlib, freeytpe, libjpeg, libpng and the other basesystem libraries that has been vulnerable. PLEASE Trolltech do not ship a phone with known security issues.

To prepare for the big changes soon coming from Richard we have decided to do two releases of BitBake. We have released version 1.4.4 from the stable branch and we have created a new stable branch from trunk that is named 1.6 and has a release as well. As with any berlios hosted project the BitBake releases can be downloaded from the Files site found at . BitBake 1.4 and 1.6 should have the same amount of features but BitBake 1.6 should be noticable faster on initial parsing. Have fun, find bugs, report them 😉

QtEmbedded has a class called QDirectPainter. This class allows to directly manipulate the mmaped screen. E.g. this can be used to memcpy images to the framebuffer like it is done in QImage, QPixmap, bitBlt. It can also be used by MediaPlayer like OpiePlayer2 to memcpy the videoplayer widget into the screen.
Having nice kernel drivers that support layers one can run the GUI on top of the video and one can control the visibility and opacity by controlling the ‘a’ of the RGBA. This can be done nicely using QDirectPainter. Just mark the relevant pixels with their alpha values and be done. The most obvious piece of code would use |= to OR the alpha value to the current pixel.

There is only bad thing with this approach. This code will do a load and store on the same pixel. As my failure showed this is not done within one or two cycles but requires more time. Now imagine you do this on a complete 640×480 display. Right you can not make the complete display transparent within one time slice but suffer context switches. The lesson learned is simply assign a value do not try to OR it

My glibc patch at http://sourceware.org/bugzilla/show_bug.cgi?id=3004 was already turned down. As Drepper says there is nothing to argue about. Building glibc requires a binary called ‘readelf’ in the $PATH and in worst case this should support the target platform. To talk debianism this means you need to have binutils-multiarch installed to configure glibc correctly. Otherwise it will fail silently and misdiagnose the features of gcc. In our case this means glibc will not be able to detect that our gcc can generate .init and .finit sections which should be what __attribute__((constructor)) and __attribute__((destructor))__ creates and every gcc >= 3.4 creates properly.

On the other hand – besides one mysterious error – I have an ARM EABI gcc4.1/glibc2.4/binutils-2.17 toolchain running on my OSX. glibc-intermediate and glibc fail when compiling some bits that look like ‘iconv’ but a second run of make works just fine. This can be an issue of using -j2 and actually having two cores or other weird issues. I will try it once again and will try hard to identify the issue. Bunutils 2.17 needed one little patch. Somehow apple’s gcc4.1 thought that char stopc in read.c might be accessed unitiniliazed. I will need to send the patch upstream which means to get another bugzilla account for another GNU project…. I think it is time for a grand unified bugs.gnu.org…

On thursday I had fun building a Qt 2.3.12 with threading statically. The new and nice configure script of Qtopia2.2 loves to create funky Makefile rules. First it compiles the software using the src-mt rule and then it decides to cd src; make clean. Which will remove everytihg named .o or .tmp and finally everything called ../lib/libqt*. This means it builds the lib, ar’s it together then removes it and fails somewhere on the missing .tmp directory when continuing to rebuild… Anyway this error was mysterious and needed three people to look into. Oh well we had the resources for three engineers…. What does that show us? There are no autobuilds of Qt running for different configurations and options. Maybe we should ask Trolltech to contribute to our tinderbox to improve their internal processes. At the end we had a 6 o 7 MB large konqueror embedded running. I think we should get that size down a bit starting to utilize visibility options and such.

On a private note it looks like Al(exandra) and me will take different directions and have split-up. I think we will get along as really good friends but only the future will tell if we start to hate each other or get back together. I wish her good luck, success and pleasure with everything she ends up doing.

I currently have access to a Intel Core Duo system with Darwin/OSX preinstalled. Having the power of two cores and a non GNU system installed I figured it is time to hack on OpenEmbedded’s support for non GNU hosts/build systems. The last time I tried this was on a G4 system and it failed miserable. It started to fail with quilt which was full of bashism and GNUism which forced me to fix the GNUism in patcher and to use that instead. It failed on flex-native, gcc, glibc, gcc (again), ipkg-native for many different reasons.

Thanks to the power of the two cores compiling, autoreconf and configure times are nice enough for the trial and error procedure. Also quilt has progressed into using less GNUism only requiring GNU sed and GNU getopt to be installed for almost working correctly. I have taken the quilt patch from fink to fix the remaining GNUism. Having quilt available is great as I will need to manage a lot of patches in the progress.

Fixing gcc was easy as well. It only has one conceptual misusage of autotools. Autoconf checks for the presence and usability of header files using CPP. In the case of gcc when building target libraries (e.g. target-libtsdc++ for arm-angstrom-linux) using /usr/bin/cpp to check for the presence of headers is not smart. e.g. libstdc++ found that I have a sys/filio.h header available. I have patched gcc 3.3,3.4,4.0,4.1 and 4.2 snapshots to use the just built compiler (xgcc) and use that as a preprocessor by adding the -E option. The result of this little patching is: The compilation of target libraries will never fail again because autoconf thinks certain features are available. This is good for people on BSD systems creating GNU toolchains and for uclibc people having less features enabled in their library. No more failing builds because host headers were used to determine target features.

Onto glibc. glibc is really mean as configure checks fail silently leading to later trouble. One big issue was the assumption of having binutils installed on my system. The configure check checked compiled bianries using ‘readelf’ to check for vertain features. I have arm-angstrong-linux-readelf installed on my system but no readelf and I won’t ever have one. The fix is easy as well. Use AC_CHECK_TARGET_TOOL([READELF], [readelf] [readelf], [$PATH]) and it will search for arm-angstrom-linux-readelf and will define $READELF within the configure. Now I will use $READELF wherever readelf was used before. Wow and now we have a compiling glibc on OSX. That was fairly easy. On FreeBSD we have one more issue as we will need to recompile the kernel to allow a large enough Command-Line.

Now I needed to introduce an evil hack into our gmp-native.bb for darwin. Apple’s gcc has certain issues with the assembly within the GNU mp code. Using none-apple-darwin as –host will force GNU mp to not use optimized but generic implementation of their tools.

And then we just compiled a full featured version of gcc 4.1.1 and it worked. At the end of the day we have a modern GNU toolchain on a non GNU system, sent patches upstream.

For the future we will fix the remaining GNUism within OpenEmbedded, make QtE and X11 cross compile nicely on OSX and we can create complete distributions on non GNU systems.

While I’m too busy to do real Free Software and OpenSource work I just wanted to break the silence and to blog again. First of all o-hand just released a nice OpenGL based canvas and from blogs I have read it was already used at GUADEC to do the presenations. I’m really looking forward to see some more offline videos from the GUADEC…

The only news directly impacting OpenEmbedded is the announcement of transition to monotone 0.27, which was finally released earlier last week. We have monotone 0.27 compiled and running on our main repository, a test conversion of the metadata (rosterify) took about 11 hours. We need to complete some scripts to verify that the migration went smooth… Well this will happen the following month.

I have bought one of the last copies of “Free As in Freedom” and I hope to find an answer to the question if RMS is doing OpenSource or not ;)… this broke the silence now I will focus on the stuff I need to complete

I have been struggling most of the day to get one my Sharks running. It is served by the NSLu2 (OpenSlug) which provides a network block device for swap and NFS for root. The Linux 2.6.17rc? kernel booted nicely got the IP address thanks to the ip=both CMDLINE specified in the OpenFirmware and wonderfully mounted the nfs and freed the init memory. And then nothing happened, nothing, just nothing. Well you can imagine how much pleasure you have in that case. I have experimented with tcp,udp,v3,v4,rsize,wsize… and all combinations, different rootfile systems. After a small chat and a common joke “Did you compile ELF support into the kernel” I decided to try another floating point emulation and then it worked! Well, I wasted some hours but now my Shark is finally booting, once again.