moiji-mobile

Hack/Product of the week, worker of the month all hail Jacob. Patching RTP sequence numbers and session id’s in RTP and RTCP to dynamically switch video streams on broadcast site without confusing players on the clients. Nice hack and see you at ITU Hong Kong.
I need to sniff the USB Communication of this dongle to get rid of Windows on the device… but there seems to be a good software tool to get this job done. Sadly the lab was closed and I can’t use hardware supported sniffing …hints?

I’m exposed to an UMPC for two days and I really hate it. I wonder how companies can sell such things. Not even a freshly liberated, booted from haret, PDA running feels that crappy.

The list of annoying things is so long and it is getting longer by second. You can not use most of the installed applications as they failed to make keyboard input working, the hardware cursors keys are not rotated when you rotate the display, dual display and touchscreen input is ridicioulously broken. If your UMPC screen is monitor 2 and you use the touchscreen you can only control the first screen. The touchscreen is not working if you switch the rotation, you end up with an offset of 10 cm from you wanted to click and where MS thinks you clicked…. It is so broken, it is so Windows.

So I have to congratulate Microsoft for bringing the windows feeling to a portable device, well done! Oh yes, I’m really annoyed.

This week I was introduced to the mini bar and enjoyed talking.

PyPy vs. smalltalk. Okay in contrast to my prior believe something PyPy is not that new. smalltalk to slang looks conceptually similiar to python and RPython. And smalltalk is implemented in smalltalk just like PyPy in python and there is a tool to create C code from the smalltalk image effectively creating the SmallTalk VM.

I should look more often into design patterns. With the new BitBake parser we build an AbstractSyntaxTree and want to evaluate it. Possible evaluations are a DOT directed graph (tree/forest), a classical bb.data dictionary or something I do not forsee, or another operation on the AST. What I created was a mixing class ASTAssigment(ASTNode,eval_factory(type.Assignment)) and this factory made sure to define an eval method. But this is completely wrong. First of all one would like to have a trait because the eval method should not modify the Tree but just evaluate. And second there is a nice pattern already it is called Visitor Pattern and basicly this is what I want. I want to visit each Node of my tree and evaluate in some way. I really should look into design patterns more often.

On a personal note I found out to have had jingoistic behaviour in the past. Note to self, do not call your future girlfriend anti-social because she uses hotmail, msn and windows and doesn’t recognize she gives up her freedom this way don’t talk with her or cope with it but don’t be jingoistic in forcing her to look into Free Software.

Emily-Style: “Get Lost, I won’t fix your computer”

DVB-H is still IP based, but the lower levels of DVB-T and DVB-H are pretty similiar. I felt bored and wrote a simple mpeg soft demuxer that can extract PES and PSI. But now when DVB-T and DVB-H are mostly compatible where is the IP? Well it is inside the DSM-CC private section inside the Multi Protocol Encapsulation (MPE). So I felt bored and wrote this multiplexer, I analyse the DSM-CC private tables and inject the payload of the MPE into the IP stack.
So besides finding a wife, having children, planting a tree I only have to write my own IP Encapsulator, and copy Fabrice and abuse video cards as DVB modulators.
Besides the ever growing KDE developer community in Berlin, the DVB group is steadily growing as well. I’m glad to announce the sith lords doku and schmid are joining our conspiracy in Berlin. All your DVB-H belongs to us *har har*. Come on you knew I’m evil.

I have finally started on the BitBake parser again, oh what a day off from everything can do, and this is completely different to the python implementation and the planned C equivalent. It is still heavily based on Marc’s Singers excellent work on Lex and Lemon (Thank you for showing me this wonderful tool) but this time everything is parsed into an AbstractSyntaxTree (AST) which we will evaluate later. Wow welcome to the wonderful world of parsing. The two constraints of this approach are. With the current parser we spend most of the time parsing the same .bbclasses over and over again. So by keeping these things as AST will help us to avoid parsing the same files over and over again. And the next constraint is to avoid expanding the bitbake data too often. At least this approach looks promising. I can parse almost all bb, inc, conf, bbclass files in about two seconds and consume only 50 megabyte of ram. So as numbers must go down we will separate parsing speed and evaluation speed. So parsing will always be fast and we will continue optimisation the evaluation. So no one can ever again complain parsing is slow. If it is slow then lex and lemon must suck, but they don’t suck.

For another A-Team like fair project we built a mosaic of Images, live h264 videos and html sites on a huge 42″ LCD panel. You have three rows with four columns and place an image, video or webbrowser widget at any position. Displaying the video and the image is quite easy. But the browser was a bit tricky.

So my first choiche was Python and KHTML. Well KHTML is attached to KIO and other services. So I found this python KDE example and changed it to display a specified url and changed KCmdLineArgs. Managing KCmdLineArgs in python was a bit tricks and documentation could have been a bit better. But we have dropped this approach because

• We had a dcopserver, kdeinint, KLauncher starting race. Sometimes instead of the site a “could not initiliaze KLauncher” error was displayed.
• It insisted on opening the Save As dialog for flash…

Okay my next try was gpe-mini-browser which uses gtk-webcore. The most amount of work was needed to make the old gtk-webcore snapshot compile on edgy. Then I had to hide the toolbar controls. Besides crashing when specifying http:// in parse_url gpe-mini-browser worked really well for the purpose and it had a good performance across the network. It displayed the binary content of the flash which was acceptable. We didn’t use this application because I did not feel like creating packages for it. gpe-mini-browser uses libcurl to do HTTP transfer the benefit is the browser does not require any other daemons running. If you want to kill it, just kill the gpe-mini-browser. In contrast to KHTMLPart and KIO this was a big win for our purpose.

Now the monster enters the game. I know that the OLPC sugar project embeds Mozilla using python. So I looked for GtkMozEmbed bindings but there weren’t any. I considered using the bindings from the SUGAR project but as I was behind a stupid firewall I was not able to git clone the laptop.org repository. So I googled for MozEmbed bindings and found the ruby bindings with an example which showed everything I needed. A bit more documentation wouldn’t hurt but the example did the stuff I needed to do so the documentation was enough. This was the first time I coded in ruby, I’m not impressed though but it got the job done. After installing ruby, irb, gtkmozembed for ruby and recognising that it misses a dep on the gtk+ bindings I was able to copy and paste the example to the ruby shell and I saw the mozilla screen.

There were some pitfalls. Specially in getting the same code running on Dapper. A magic Gtk.init was missing to make the ruby script run on dapper but now our mosaic is powered by ruby and mozilla.

I would have preferred to use KHTML/WebKit but the time might not be ready. With WebKit and Qt4 and Curl we might get there. But sadly the cmake System of WebKit heavily depends on KDE trunk being installed. This needs some more fixing.

Wow, what a day. Today was the “Conference on open-source mobile phones mobilizes”. And OpenMoko here and here was announced. Also the OpenMoko.orgwebsite lacks the glamour of the usual announcements it has something true geeks will love. It has Web SVN, and yes it contains the source for all these things they announced and even datasheets. To use a sentence the Qt Community Manager uses a lot. FIC and OpenMoko.org just get it. They know what geeks like and need. It is a Open Platform, build by people that care about Free Software. I think both Harald Welte and Mickey have a track record in understanding Free Software. Hey guys this is so great and it really helps Free Software, Linux and Gtk+ to move forward. This device and software is built with Free Software in mind this means they are GPL conform from day one, no hassle to wait weeks or month for the sourcecode of your toolchain, kernel just to fix a bug. This really makes a huge difference. Keep on rocking and I owe you Club Mate, alcoholic beverage!

I think OpenMoko is unmatched among all Linux Mobile platforms movements because it is build from community members and Free Software enthusiasts. And they focus on the important bits creating a platform and providing the basic set of applications and hopefully they will follow the release early and release often scheme! Let us make Free Software go forward!

Happy Birthday lovely Qtopia. It is another year you are truely proprietary. The last GPL release, patch release rsync version of Qtopia was one year ago. One year without a patch, update and only five to six releases of the commercial edition. Anyway Qtopia is still my all time favorite of proprietary software and I still believe there will be a GPL release of Qtopia in the future and I hope soon really means soon.
But for now happy birthday lovely proprietary piece of software and for the future I hope your bytes will be Free.

Determining the Quality of OpenEmbedded is quite hard as we depend on 4000 pieces of software. So OpenEmbedded quality is aggregated from the quality of all such independant projects. To make things worse we use these things in many different combinations. By guess we have +10 target distributions +30 target machines, +4000 package/build descriptions and we should work on all available Linux distributions and somehow even non GNU UNIX platforms. I don’t want to do the math to think about each possible combination but trust me this is more that we currently have computing power to compile. Then people would need to run these apps and their tests on target platforms and some how all these results must me managable.

The idea on how to test it and manage quality. The main strategy is to track difference and notice early when things break, notice them before a user would have a chanche to notice. The most common issues are missing source, failed configure, failed builds or wrongly packaged.

• Test if this application/image in this configuration with that toolchain properly compiles and links. We can track binary size differences, the build times and monitor differences.
• Test the packaging. Test if some files are in a package where they shouldn’t be. This applies to .so files or .debug directories. Also test .la and .pc files if they point to the right directories. Check if the architecture of the file is compatible with the TARGET.
• Test -native packages after we have compiled them and before we stage them. Run the tests suits of these applications and refuse to work when they are broken. This is tricky as there are sometimes known issues, e.g. with gcc where a lot of tests will fail but are known to fail. So we can only fail if we really have an unexpected failure.
• The real nice thing to test is if this application actually works. It may not work because this is a known software issue or we have done something wrong or the application makes bad assumptions on the target architecture. But we need test cases for these things. Many of the GNU utilities have something to test them, these testcases must be able to be packaged and execute on a phone using the installed software. This is actually tricky as well. If we manage to get the testcase to execute on the target platform we still need to get the results back. With DogTails
• Track test coverage of the software we build. Always report how many lines are untested.

The architecture to solve some of these issues consists out of patches to our metadata and bbclasses. E.g. the “Cross Compile Badness” check which looks at some known wrong include directories when cross compiling and aborts. This way applications will fail when they have -I/usr/include in their INCLUDEPATH. Packagers will be made aware of this issue when they create the package and add it to OpenEmbedded. Also with the insane BitBake class we will be able to see the following issues: insecure RPATH in the binaries, wrongly packaged files, packages where ARCH and content are different, wrong depends. Together with BitTest we will be able to see undocumented keys, missing sources, wrong RDEPENDS for -native packages. I think we slowly get to the point where we have the tools to secure the quality of our OpenEmbedded product.

On the server side we are currently lagging behind. We have the crappy tinderbox3 from the Mozilla Foundation but I have started writing a new one using python and Django. This setup will help us to relate Bugtracker, IRC, Build- and Testresults. You will be able to do queries like. Show me everything about the package gimp-foo. You will be able to see test results, compilations, you will be able to filter for weeks/month/architecture.

But why is this important? A complete setup will allow us to spot regressions fast, inform the developers immediately, e.g. through RSS feeds or IRC. It will allow to see which software works in which configurations so it will be possible to declare known good configurations. E.g. compile servers could make their images available which get regularily booted and ran on QEMU devices for ARM, MIPS, PowerPC and other real devices which send back reports to the system, e.g. including coverage data and memory footprints. So we will always know that the applications compiled and ran. We only need the discipline to add tests after having found issues and encourage the projects we compile to create test suites themselves.

Oh and finally it is fun to create and setup such a system. The current status is okay as well. We have a Bonsai system which relates to Bugtrackers. Going to here you will be able to do simple queries. If a string like fix #123 is found you will be redirected to the bug tracker. This bonsai will be used to relate build results to changes in the SCM.

Imagine we track all the SCMs we build against. Imagine you build the gimp and the memory footprint during the tests increased by two megabytes and you want to track it down. You will be able to see all the changes made to the dependencies of the gimp between this build and the last known one. You should be able to find the source of this issue quickly.
For the tinderbox itself I have written a first model which I will try to fill with some data to find out how it performs. Actually I’m really excited about how well these things can be integrated.

There are two aspects of Software Engineering that currently interest me most. One is taking Usability into account when creating applications. What tasks needs to be solved, who is our target user, how often are these tasks executed. All these questions will heavily influence the User Interface and behaviour of your application. I started as a geek adding geek features to applications. So dear lazyweb and specially Free Software developer. If you write a new application ask yourself what these primary usecases are, talk with usability experts as they will help you to write better software. If you plan adding this very important feature to an existing application, please ask yourself how this fits with the primary usecase of the application and the target user.
No the addressbook for your grandmother doesn’t need easy access to the GPS coordinates of her weapons of mass destruction. Such patches and features wishes should always be ignored. If you are in doubt if such a feature is appropriate consult an expert. We are experts on technology and sometimes by accident we create an usable userinterface and this brings me to another subject, one where I have more domain knowledge.

Quality and Quality Control in Free Software projects. The important question is always what is quality? What is good quality? Technically I always try to achieve the perfect quality but from a QA point of view this term is undefined. The Software we create should not crash, it should not have security issues and you should not lose data. I think this is mandantory quality that everyone agrees on. But what about questions such as usability? Can this application be used to do task X and how easy it is to get task X done. Do people know that task X can be solved using this way or do they do it in a more complicated way?

This leads us to my main question? How to check and assure the quality of your software? Regression tests, memory usage, coverage tests are well known types of tests. They are a good a tracking the quality of your software but these are only on a technical level. They answer questions like does this application crash, leak memory, or loses data again? But they don’t answer the really important question. Can this application be used by our target and will he be happy? I have no idea how we could test this though.

h.

Hey Lorn,
I’m not going to abuse blogs as mailinglist or forum but I have some random thoughts. So Lorn Microsoft is a Free Software supporter because they hired the Gentoo Founder? I always thought that Microsoft doesn’t like Free Software too much, after applying your logic I recognize my failure. I like old cars as well, I used to drive an old VW beatle during summer and winter. Yes it was old, easy to repair and I think I understood most parts of the car. Oh and it has had serve security issues as well. But in contrast to cars, software isn’t likely to stay the same for more than fourty years and as our products are virtual I prefer to fix security issues. I prefer to fix issues that annoy users, I prefer to fix issues where a user can lose personal data. So in contrast to my old VW Volkswagen Qtopia 2.2 is not usable for many reasons. I get spare parts for the beatle, I do not get support for Qtopia 2.2, fabricating issues of the beatle are well documented, they are not documented at all for Qtopia, there is a local community of beatle drivers helpimg me out in need… I think if I would have a commercial Qtopia2.2 license and file bug reports your support will likely to tell me that these things are fixed in Qtopia4 and ask me to upgrade. So even your support will ask me to not use the unmaintained Qtopia2.2.
And there is no word in my previous post that blames Qtopia for something. Opie’s error was the lack of differentiation to Qtopia. People thought both were Free Software and failed to see the difference. The difference was that Opie secured the Freedom of the Platform and created this community were issues were documented and fixed. This included public mailinglists, bugtracker and source control management.

And to the compability. I seriously wonder if you are a core developer, and you found issues why didn’t you file bug reports? Or as you are a core developer why didn’t you write a fix and shared it with us? This was what Max and Me did when making Opie-Console work for the Sharp.ROM. Do you know about weak symbols? These things make it possible to compile apps against Opie’s libraries and let them work on Sharp.ROM and Qtopia1.7. So really if you don’t have a clue about how our and Qtopia’s library work internally don’t make false statements.

Oh and yes, shared libraries are fluff. Instead of sharing an implementation between 15+ apps it would be better to write 15 different implementations with different bugs and different issues. And if you don’t see the value of a sane and flexible debugging framework talk to the creators of Qt and ask them why they did something similiar in Qt4. Have you ever debugged on a device? And yes getting debug messages across a network is a feature I have regulary used when debugging on devices. This can be used when running test cases, it can be used when you don’t have much space on rootfs and don’t want to use NFS for keeping logs. But such features are only obvious if you actually test stuff on devices.

love and peace
zecke

PS: luckily I didn’t respond