C++ – moiji-mobile

QtVirtualKeyboard on Wayland

2017-11-26

For the last couple of years my focus was on the Osmocom project to bring Free Software to the world of telecommunication. With a group of enthusiasts we have implemented the components necessary to run a complete network using Free Software. The Rhizomatica project is using the software to connecting people that were left behind. Our tools enabled high impact security research leading, leading to improvements to privacy and security for all of us….

But during the last months I had the opportunity to return to C++ and Qt work and it feels like coming home to the world of ARM powered hardware. When I left, the transition from consumer electronics (e.g. settop boxes) to automative (e.g. IVI) began and it seems it successfully completed! On Friday I explored a regression in OpenSSL and today I had the pleasure to understand input method handling of wayland a little bit better.

I wanted to see if I can use wayland and run QtVirtualKeyboard only in the Compositor. I couldn’t find answers in the documentation and started to read the code. Once I understood how it should work, I found a simple example in QtWayland. Isn’t Qt wonderful?

As I have gone down the rabbit hole let me try to share some of my reading. At the core is the unstable (API subject to change) text-input protocol. Upstream of wayland-protocols is still at v1 of the protocol but QtWayland includes a copy of v2 and has rather complete support for the compositor and client (thanks KDAB!).

Compositor/Server

To make it work the compositor needs to signal support for the “zwp_text_input_manager_v2” interface. In QML/Quick this needs to be done by adding the following inside the WaylandCompositor component:

...
TextInputManager {
}
...

This will instantiate a QWaylandTextInputManager and call ::initialize on it. Then auto-generated code will use wl_global_create to register the interface in the global registry. Clients will request a text_input and the manager will create one or use a QWaylandTextInput and then send events down to the client. E.g. if there is a soft keyboard set a breakpoint in QWaylandTextInput::sendKeyEvent in the compositor and see from where it is coming and how it will send the event to a client.

Client

Once the client starts the QWaylandIntegration will create a QWaylandDisplay and then query/initialize the registry. The answer will call QWaylandDisplay::registry_global for every registered interface. For the the zwp_text_input_manager_v2 interface the integration will initialize a QtWayland::zwp_text_input_manager_v2 and for each input device a QtWaylandClient::QWaylandTextInput (different namespace) will be created.

Communication

Now Compositor and Client will communicate through the wayland connection (no dbus). With Qt APIs the client will notice which element has been focused and will request the input method to be shown (QtWaylandClient::QWaylandInputContext::showInputPanel) and the Compositor will forward this to the QInputMethod (see QWaylandTextInputPrivate::zwp_text_input_v2_show_input_panel)

Putting it together

In the end this is rather simple.. create a TextInputManager, use the InputPanel of the QtVirtualKeyboard and start your compositor with QT_IM_MODULE=qtvirtualkeyboard. Done!

Connectivity options for mobile M2M/IoT/Connected devices

2016-07-24

Many of us deal or will deal with (connected) M2M/IoT devices. This might be writing firmware for microcontrollers, using a RTOS like NuttX or a full blown Unix (like) operating system like FreeBSD or Yocto/Poky Linux, creating and building code to run on the device, processing data in the backend or somewhere inbetween. Many of these devices will have sensors to collect data like GNSS position/time, temperature, light detector, measuring acceleration, see airplanes, detect lightnings, etc.The backend problem is work but mostly “solved”. One can rely on something like Amazon IoT or creating a powerful infrastructure using many of the FOSS options for message routing, data storage, indexing and retrieval in C++. In this post I want to focus about the little detail of how data can go from the device to the backend.

Image from Wikipedia (CC BY-SA 3.0 by Herzi Pinki)

To make this thought experiment a bit more real let’s imagine we want to build a bicycle lock/tracker. Many of my colleagues ride their bicycle to work and bikes being stolen remains a big tragedy. So the primary focus of an IoT device would be to prevent theft (make other bikes a more easy target) or making selling a stolen bicycle more difficult (e.g. by easily checking if something has been stolen) and in case it has been stolen to make it more easy to find the current location.

Architecture

Let’s assume two different architectures. One possibility is to have the bicycle actively acquire the position and then try to push this information to a server (“active push”). Another approach is to have fixed installed scanning stations or users to scan/report bicycles (“passive pull”). Both lead to very different designs.

Active Push

The system would need some sort of GNSS module, a microcontroller or some full blown SoC to run Linux, an accelerator meter and maybe more sensors. It should somehow fit into an average bicycle frame, have good antennas to work from inside the frame, last/work for the lifetime of a bicycle and most importantly a way to bridge the air-gap from the bicycle to the server.

Push architecture

Passive Pull

The device would not know its position or if it is moved. It might be a simple barcode/QR code/NFC/iBeacon/etc. In case of a barcode it could be the serial number of the frame and some owner/registration information. In case of NFC it should be a randomized serial number (if possible to increase privacy). Users would need to scan the barcode/QR-code and an application would annotate the found bicycle with the current location (cell towers, wifi networks, WGS 84 coordinate) and upload it to the server. For NFC the smartphone might be able to scan the tag and one can try to put readers at busy locations.

The incentive for the app user is to feel good collecting points for scanning bicycles, maybe some rewards if a stolen bicycle is found. Buyers could easily check bicycles if they were reported as stolen (not considering the difficulty of how to establish ownership).

Pull architecture

Technology requirements

The technologies that come to my mind are Barcode, QR-Code, play some humanly not hearable noise and decode in an app, NFC, ZigBee, 6LoWPAN, Bluetooth, Bluetooth Smart, GSM, UMTS, LTE, NB-IOT. Next I will look at the main differentiation/constraints of these technologies and provide a small explanation and finish how these constraints interact with each other.

World wide usable

Radio Technology operates on a specific set of radio frequencies (Bands). Each country may manage these frequencies separately and this can lead to having to use the same technology on different bands depending on the current country. This will increase the complexity of the antenna design (or require multiple of them), make mechanical design more complex, makes software testing more difficult, production testing, etc. Or there might be multiple users/technologies on the same band (e.g. wifi + bluetooth or just too many wifis).

Power consumption

Each radio technology requires to broadcast and might require to listen or permanently monitor the air for incoming messages (“paging”). With NFC the scanner might be able to power the device but for other technologies this is unlikely to be true. One will need to define the lifetime of the device and the size of the battery or look into ways of replacing/recycling batteries or to charge them.

Range

Different technologies were designed to work with sender/receiver being away at different min/max. distances (and speeds but that is not relevant for the lock nor is the bandwidth for our application). E.g. with Near Field Communication (NFC) the workable range is meters while with GSM it will be many kilometers and with UMTS the cell size depends on how many phones are currently using it (the cell is breathing).

Pick two of three

Ideally we want something that works over long distances, requires no battery to send/receive and the system is still pushing out the position/acceleration/event report to servers. Sadly this is not how reality works and we will have to set priorities.

The more bands to support, the more complicated the antenna design, production, calibration, testing. It might be that one technology does not work in all countries or that it is not equally popular or the market situation is different, e.g. some cities have city wide public hotspots, some don’t.

Higher power transmission increases the range but increases the power consumption even more. More current will be used during transmission which requires a better hardware design to buffer the spikes, a bigger battery and ultimately a way to charge or efficiently replace batteries.Given these constraints it is time to explore some technologies. I will use the one already mentioned at the beginning of this section.

Technologies

Technology	Bands	Global coverage	Range	Battery needed	Scan Device needed	Cost of device	Arch.	Comment
Barcode/QR-Code	Optical	Yes	Centimeters	No	App scanning barcode required	extremely low	Pull	Sticker needs to be hard to remove and visible, maybe embedded to the frame
Play audio	Non human hearable audio	Yes	Centimeters	Yes	App recording audio	moderate	Pull	Button to play audio?
NFC	13.56 Mhz	Yes	Centimeters	No	Yes	extremely low	Pull	Privacy issues
RFID	Many	Yes, but not on single band	Centimeters to meters	Yes	Receiver required	low	Pull	Many bands, specific readers needed
Bluetooth LE	2.4 Ghz	Yes	Meters	Yes	Yes, but common	low	Pull/Push	Competes with Wifi for spectrum
ZigBee	Multiple	Yes, but not on single band	Meters	Yes	Yes	mid	Push	Not commonly deployed, software more involved
6LoWPAN	Like ZigBee	Like ZigBee	Meters	Yes	Yes	low	Push	Uses ZigBee physical layer and then IPv6. Requires 6LoWPAN to Internet translation
GSM	800/900, 1800/1900	Almost besides South Korea, Japan, some islands	Kilometers	Yes	No	moderate	Push	Almost global coverage, direct communication with backend possible
UMTS	Many	Less than GSM but South Korea, Japan	Meters to Kilometers depends on usage	Yes	No	high	Push	Higher power usage than GSM, higher device cost
LTE	Many	Less than GSM	Designed for kilometers	Yes	No	high	Push	Expensive, higher power consumption
NB-IOT (LTE)	Many	Not deployed	Kilometers	Yes	No	high	Push	Not deployed and coming in the future. Can embed GSM equally well into a LTE carrier

Conclusion

Both a push and pull architecture seem to be feasible and create different challenges and possibilities. A pull architecture will require at least Smartphone App support and maybe a custom receiver device. It will only work in regions with lots of users and making privacy/tracking more difficult is something to solve.

For push technology using GSM is a good approach. If coverage in South Korea or Japan is required a mix of GSM/UMTS might be an option. NB-IOT seems nice but right now it is not deployed and it is not clear if a module will require less power than a GSM module. NB-IOT might only be in the interest of basestation vendors (the future will tell). Using GSM/UMTS brings its own set of problems on the device side but that is for other posts.

C++, Qt and Treefrog to build user facing web applications

2016-05-07

In the past I have written about my usage of Tufao and Qt to build REST services. This time I am writing about my experience of using the TreeFrog framework to build a full web application.

You might wonder why one would want to build such a thing in a statically and compiled language instead of something more dynamic. There are a few reasons for it:

Performance: The application is intended to run on our sysmoBTS GSM Basestation (TI Davinci DM644x). By modern standards it is a very low-end SoC (ARMv5te instruction set, single core, etc, low amount of RAM) and at the same time still perfectly fine to run a GSM network.
Interface: For GSM we have various libraries with a C programming interface and they are easy to consume from C++.
Compilation/Distribution: By (cross-)building the application there is a “single” executable and we don’t have the dependency mess of Ruby.

The second decision was to not use Tufao and search for a framework that has user management and a template/rendering/canvas engine built-in. At the Chaos Computer Camp in 2007 I remember to have heard a conversation of “Qt” for the Web (Wt, C++ Web Toolkit) and this was the first framework I looked at. It seems like a fine project/product but interfacing with Qt seemed like an after thought. I continued to look and ended up finding and trying the TreeFrog framework.

I am really surprised how long this project exists without having heard about it. It is using/built on top of Qt, uses QtSQL for the ORM mapping, QMetaObject for dispatching to controllers and the template engine and resembles Ruby on Rails a lot. It has two template engines, routing of URLs to controllers/slots, one can embed any C++ in the template. The documentation is complete and by using the search on the website I found everything I was searching for my “advanced” topics. Because of my own stupidity I ended up single stepping through the code and a Qt coder should feel right at home.

My favorite features:

tspawn model TableName will autogenerate (and update) a C++ model based on the table in the database. The updating is working as well.
The application builds a libmodel.so, libhelper.so (I removed that) and libcontroller.so. When using the -r option of the application the application will respawn itself. At first I thought I would not like it but it improves round trip times.
C++ in the template. The ERB template is parsed and a C++ class will be generated and the ::toString() method will generate the HTML code. So in case something is going wrong, it is very easy to inspect.

If you are currently using Ruby on Rails, Django but would like to do it with C++, have a look at TreeFrog. I really like it so far.

A C++ project without Qt

2016-01-03

My primary language is Smalltalk (Pharo and GNU Smalltalk) and I didn’t write much C++1x code yet and for a new project I decided to use C++ and experiment. The task was rather simple, receive a message, check if this message type needs to be re-written, start to parse it, make a MongoDB look-up, patch it, send it.

While looking for MongoDB drivers for Qt I only found the pure C++ driver that is working with std::string, exceptions and only offers a blocking interface. At this point I decided to see how painful using stl is these days and opted for a pure stl project. Many years ago the only online documentation came from SGI, there were no examples, no autocompletion in editors, etc…

The application is relatively trivial. It is using lambdas to hook two components (the receiver and the patcher) together, it is using std::thread with lambdas to spawn worker threads because of the blocking interface of the database driver, std::chrono for some ad-hoc StatsD code to add event counters and performance monitoring.

So how did this experiment go?

The C++11/C++14 documentation that is available has improved over the old times, it is still below the quality of the Qt documentation.

Once deploying this on Debian I immediately had odd crashes and I blame ABI issues as my application used C++1x but the mongo-cxx-driver was apparently not using it. I ended up packaging the latest stable (“legacy”) standalone driver for debian and made sure it is compiled with C++1x.

The std::chrono API has potential (as it allows me to control the resolution) but the API, specially in C++11 without the new literals, makes easy things difficult. All I wanted was something like QTimer::elapsed() or QElapsedTimer::elapsed(). So I think the Qt API is better designed for this usecase (my measurement resolution is milliseconds and not picoseconds).

For socket code I was using pure C/libc. As my database code was blocking I could make my socket code blocking as well.

I normally use vim but when working with an unknown API I totally enjoy the comfort QtCreator is providing. I was able to jump to the header files and explore the interface. I think it has saved me quite some time while wrangling with the stl.

New languages come with their own build tools these days (Rust/Cargo, Ruby/Rake, …) and C++1x doesn’t have anything to offer here. The options I considered where autotools, CMake or qmake. I think autotools are overkill here, I dislike CMake for various reasons and I like the simplicity of qmake.

In the past I had played with the Google C++ Unittest framework and actually liked it a lot. These days one is asked to copy and paste the framework into the code base. It is an ongoing struggle if one should embed third party libraries or not but as my application is rather small I decided it is a no go. I have used and enjoyed the Qt testlib. So my STL only application is tested using QObject and QCoreApplication.

Conclusion:

The STL (and the documentation) has improved. I still prefer the API design of Qt and when not just interfacing with an library that is using std::string Qt is still my first choice (as it provides event loops, signals/slots, networking, http, etc.).