Device profiles in Qt5

Device profiles in Qt5

OpenGL and Devices

The future of Qt’s graphic stack is OpenGL (ES 2.0), but this makes things more complicated in the device space. The library names and low level initialization needed for OpenGL is not standardized. This means that for a given board one needs to link libQtGui to different libraries and one needs to patch the QPA platform plugins to add device specific bits. The GPU vendor might provide DirectFB/eGL integration but one needs to call a special function to bind eGL to DirectFB.

Historic Approach

The historic Qt approach is to keep patches out of tree, custom mkspecs files that need to be copied into Qt before building. I have had two issues with this approach:
  1. Device support should be an essential part of Qt5.
  2. Build testing (and later unit testing) is more complicated.

Device Profile Proposal

The approach we took is a pragmatic one, it should be easy for device manufacturers to do the right thing, it should not be a burden for the maintainability of Qt. After some iterations we ended up with device profile proposal and began to implement to it. Most of it is merged by now.

Key Features

It begins with the ./configure script that now has the -device=DEVICE and -device-option KEY=VALUE to select a device and to pass options, e.g. additional include paths, BSP package, to Qt. The second part is a device to influence the behavior of QPA platform plugins. Right now this applies to the DirectFB and EGLFS plugin. A device can install hooks that are called as part of the initialization of these plugins. The hook is the pragmatic approach to get a mix-in with the existing code base.

Supported Devices

Right now we have completed device support for the RaspberryPi,BCM97425 and AMLogic 8726-M. We do support some more devices but they might still require external patches.
Cross compiling Qt and dealing with teams

Cross compiling Qt and dealing with teams

Let me share some notes on how Embedded Linux programming can be, if you decide the war/pain is over. We do this by looking at the desktop build and then look at the tons of stuff that you need for embedded programming.

Desktop

$ pkg-config –cflags libpng12
…
$ qmake && make
…

Embedded
$ . /usr/local/angstrom/arm/environment
$ pkg-config –cflags libpng12..
…
$ qmake-qt4 && make

Where is the difference?
There is little, you just use a cross compiler. The GNU/Linux Toolchain is prepared for cross compilation, pkg-config works with cross compilation.

What if I want different options
You can decide to use different -mtune and -march values, or you can easily regenerate the SDK targeting a different CPU architecture or just the instruction set level of it.

How is this possible
One creates a level of abstraction. The question is why do you want to suffer and don’t use the abstraction known from the desktop? The above is given to you by the Yocto/OpenEmbedded and details can be found in the OpenEmbedded Manual. Read here for more details on creating a Qt Toolchain/SDK and using it.

But I want to recompile Qt
Maybe you want to change the qconfig.h, or you have a bug in Qt where you want to test things more interactively. You could either do this in the build directory of Qt of Yocto/OpenEmbedded but then again the created SDK contains all the dependencies for building Qt. All that is missing is the generic qmake spec (that takes its values from the environment) that would need to be copied into mkspecs.

But my application needs libpainful-to-build
The SDK is deployed from header files of packages, you can install more -dev packages into your toolchain. If you use any of armv4t, armv5te, armv6, armv6-nofp, armv7, armv7a, … chances are high that there is already a build for your machine…

So why is it so painful?
I really don’t know, tell me.

Debugging hints I

Debugging hints I

Being an Engineer one need to resolve problems and sending a mail to a mailinglist and asking for help is most of the time not the problem solving skill you should use. So here is a list of some easy hints…

  • Finding a build error (using make) in a log file. You might use a tool like OpenEmbedded that keeps the log file around and you have a build failure. First of all check the end of the file, but one might have compiled with -j X and the failure was a bit earlier. The best way to find the error is to search for ‘***’. E.g. inside or less or vi type (/***) and it will bring you to the first compile error… GNU make is kind enough to flag the error.
  • The compiler is bitching about something that might not make any sense. With C languages a program called CPP, the C PreProcessor, is ran over your code and it might include funny headers changing your code. What I normally do is to use the compile line as seen in the terminal and replace the -c with a -E as this will only preprocess and then look at the code. The pre processed code is long, search for includes from places you don’t expect and such…
  • If a program points you to a log file, read it. In the case of autoconf (configure) a config.log is written, now the bad news is that the error is not at the bottom of the file. What I do is to look at the text of the last test and search for that inside the config.log, it brings me to the failing test…
  • Dealing with remote crashes. You have a segfault but don’t know when/where it is happen and it is hard to reproduce? Just use ulimit -c unlimited in a shell and run your script/applications, the kernel will place a core dump in the directory and you can use gdb and your binary to inspect it.
  • Running a libtool based binary/lib from inside the build directory. You can use libtool –mode=execute gdb tests/sms/sms_test and then libtool will setup the environment and invoke your command..

more hints at a later time…
Using oeaudit.py

Using oeaudit.py

In the last days I have cleaned up my OE Audit tool and it should be usable by everyone now. The tool requires two inputs, one is the list of packages to be build by OE for the given configuration (distro, machine) and the other is the FreeBSD auditfile. This FreeBSD auditfile can be automatically downloaded.

Without much more overhad, here we go.

$ bitbake -s > available
$ export PYTHONPATH=/place/bitbake/lib
$ /OE/contrib/oeaudit/oe_audit.py -f
This will fetch the auditfile from the FreeBSD project for you
$ /OE/contrib/oeaudit/oe_audit.py -a auditfile -p available
Now you will see a list of vulnerabilities in the packages OE is going to use.

Dealing with security issues in the context of OpenEmbedded

Dealing with security issues in the context of OpenEmbedded

One thing that has bothered me while being at Openmoko is the lack of Security Response by the OpenEmbedded Crew. In one way a security issue is just like any other bug and distros don’t upgrade each package for each bug fixed upstream but it is getting worse when the security issues exists in the default installation, in a daemon listening to network traffic and such with ready to get exploits on the network.

I think it is really unethically to go around and claim how great OpenEmbedded is and then companies like Openmoko, Palm, etc. ship vulnerable software to their users and it is easy to pass the black pit to companies actually using OpenEmbedded, let me say it is too easy.

There are various things one can do to address these problems. One option is to downgrade and use the classic Buildroot as their maintainers seem to address vulnerabilities in time. I use the word downgrade as these systems provide less functionality, flexibility than OpenEmbedded, e.g. they lack the creating of SDKs, chosing the libc (glibc, uclibc, eglibc) but then again they do their homework and provide people with security updates in time, the other option is to go to a distribution like Debian or Fedora with a proven track record of handling security issues.

But I’m going to talk about the third option that includes improving OpenEmbedded. I had the idea while being at Openmoko but the guy who was assigned to do this was laid off shortly afterwards so it never happened. In general for every package we ship in OpenEmbedded there is an established distribution (e.g. FreeBSD, Debian, Fedora) that is shipping it as well. Or in the seldom cases where OpenEmbedded is the first adopter, the software is kept current and there is not much security research anyway. This means that to provide security upgrades to our users we only need to monitor the big and established guys and that sounds like something that can be partially scripted.

I’m using FreeBSD on my servers and in the FreeBSD world there is an application called portaudit which is looking at your built/installed packages and is comparing the name, package version and patch release to a list of known security issues in the ports tree and then asks you to upgrade, Gentoo has a similiar XML file for each security incident, Debian has a security feed as well.

A long story short, on a flight to iceland I was hacking a python script called oe_audit.py that is using the FreeBSD auditfile and the output of “bitbake -s” (the list of providers and their versions) and then starts comparing these lists. Right now the script is inside the OE tree, it is still a gross hack but I will improve it to be a proper python script. In its first version it has found issues with plenty of packages in OpenEmbedded and thanks to the help of some we are down to only a couple of issues in our tree. In general addressing security issues is not that hard, follow a couple of mailinglists, look at websites, when a CVE is published search for the patch, apply it to our version, be done. Specially given the amount of OE developers we could nominate a security sherif each week that has to do the upgrades… It is not that we see more than three upgrade a week anyway… So this week it would have been Pango, Php, Pulseaudio…

On getting Free Support

On getting Free Support

Most of the things I know about Software and Hardware I have from reading books, looking at sourcecode but most importantly people willing to answer my questions on IRC and giving me a direction I could look for answers.

Now it seems to be my part to give back and help others to gain knowledge, but some things have changed. Free Software has made it to the mainstream, so besides hackers that want to understand things, we do have paid programmers that don’t want to understand but still need to make it work.

So the other day I was finding me in a IRC query. His target board was a Freescale i.MX27 ADS and his mission was to make it boot from NAND. He was using the OpenEmbedded “mx27ads” machine and had successfully build a kernel and rootfs. Now the problem is that Freescale is not particulary liked in the Free and Open Source Software Community and that Freescale prefers to be on an isolated island. After about 8 hours of spending my time on this, i decided to get back to paid work and carry on.

If you are searching for support on mailinglists, irc channels and irc queries be prepared to think, giving Free Support means that you will be helped to understand the problem and have to pick a solution yourself. If you don’t like that, don’t want to think, don’t have the time to think, you should consider getting someone from the irc channel as consultant.

So here is a list of things that work when paying a consultant but not when you are paid to do your job and you need someone else to do it for you:

  1. Pasting your log somewhere and then ask what is wrong with it. In case of compile failures with GNU make you have to search for ‘***’ in the error log, in case of an early failure of a bitbake run it tells you what software to install, in case of configure failures read the config.log, in case of other failures search the log for ‘Error’, ‘rror’, ‘Failure’, ‘ailure’. And most importantly think before you post it.

    At OpenEmbedded we check for Software being installed on the system. This includes checking for GIT, CVS, SVN and other projects needed to bootstrap. If you don’t have it installed OpenEmbedded will tell you “Error… you don’t have installed:” and the output will be finished by “You will have to install….”. If you paste such an error and ask what the problem is you are really embarassing yourself, your parents, your teachers and your country. The Error message tells you which binaries it searched for and couldn’t find, and it points you to common package names for them. I don’t think it can be any easier for a Software Engineer.

  2. Knowing your hardware. Remember, you have the hardware in front of you, you have it connected somewhere, you have something compiled on your system. So if you are asked about the system specification you will have to point to a PDF, website that names the SoC, the used flash chip (NAND, NOR, erase size, name of it), the SDRAM used and whatever else is used on the system. If you paste a FAQ of something like Buildroot you have clearly failed. Know the stuff that is on your desk, if you don’t nobody can help you.
  3. Pasting unrelated information. When you are asked to paste the output of something. Do not just write that oneline by hand. In many cases we humans apply interpretation to things. When you are asking for help it is an indication that you are not able to interpret the result in a way that lead to a result. Paste the full log, it is saving everyone a lot of time.
  4. Listen/Read to what people tell you. If you have a kernel without NAND support, but want to boot from NAND. You will have to enable NAND and the MXC NAND driver in your kernel. You will have to compile the kernel again, and you will need to boot that kernel. Now we are all humans and have done mistakes before.

    One of the most common mistakes is to not change the config, or not to boot the kernel you have built.

    For the config, check .config after you have built if it is the one you expected it to be, in case of OpenEmbedded copy the .config back to the defconfig (recipes/linux/${PN}-${PV}/${MACHINE}/defconfig). You should copy it back to have a backup in the case you are rebuilding the kernel or such.

    For the second thing it is rather easy. The kernel does contain the TIME it was built and it contains the number of times you have built it from the same directory. So if you rebuild the kernel the TIME and the NUMBER will go up. The kernel does print this information at bootup. If you are asked to check that, don’t paste the log, but check it for yourself. Last but not least, if the NUMBER did not go up or the TIME did not change you have either not rebuilt it, or not from the same directory… And if you didn’t rebuild from the same directory then you are likely to not use the right config…

  5. Don’t do crazy things while people spend their time to help you. Do not remove your build directories and just start over. No, even after your next rebuild the kernel will lack NAND support. That is because it is not enabled in the defconfig for your machine, and the build process is deterministic… You have someone on the other side that decided he wants to help you, if you are not focused, why should he?

And for all of you, that have read until the end. If you decide to seek help in an open forum. First do your homework by having some idea about the problem, have information ready (hardware spec, build logs, whatever), be prepared to think, formulate a hypothesis and try it.

With Free Software you are in the fortunate situation that you can talk to the guys who build the stuff you are using, all you need to do is to be focused on receiving help and be prepared to think.