Coffee Space


Listen:

Linux PDA

Preview Image

Preview Image

Problem

Recently Pine announced the new PineNote device, a powerful open-source e-ink device that is more powerful than the PineTab - by a lot! Unfortunately, due to patents, the device will be $400. Whilst still a massive bargain for this class of device, this is greatly outside of my budget.

But this got me thinking, how would I use this device? Is it really the device I actually want?

There have been several discussions recently over on the Pine forum about devices that I am very interested in. In one post there is a discussion about an e-paper variant of the PinePhone. For a while now we have been taunted with the prospect of a PineCom device, but I really suspect this will not be realised for quite some time as yet. I'm sure there are other discussions by members of the Pine community too...

Many of these posts are simply just ideas, or are projects with no real expectation of completion.

Often I find myself wanting a small, portable, low-power Linux device that just does one thing, really well. I want to be able to read my emails, check a web page, make a note, do a calculation - nothing too computationally heavy. Just a series of very simple tasks, one at a time. This seems like a reasonable expectation, and yet this is actually quite hard to find.

High-power solutions exist, such as mobile phones. If you're happy with large binary blobs that side load applications that cannot be deleted, or are willing to pay an arm and leg, then these are great. But mobile phones are almost too capable, they are quite literally mini-desktop machines. I have no doubt there are now mobiles out there more powerful than my daily driver desktop machine!

Worth a mention here is Pine's PinePhone, but as it's called a "smart phone", people expect it to do smart things. It has 4 cores, 3GB of RAM, eats battery, has several cameras, full-fat Linux and a mostly closed-source cell modem.

There are of course low-power solutions, based on the BL602 or the near-infinite devices based on Espressif's ESP8266/ESP32. The problem with these low-power solutions is that they are slightly too limited and cannot build upon the many existing applications and drivers that already exist for Linux. Sure, you could make it do one of these tasks - but you're not going to get many different applications working seamlessly all on the same device.

(Old) Solution

What I guess I'm really looking for is a modern-day PDA. As Wikipedia describes it:

A personal digital assistant (PDA), also known as a handheld PC, is a variety mobile device which functions as a personal information manager. PDAs have been mostly displaced by the widespread adoption of highly capable smartphones, in particular those based on iOS and Android.

Nearly all modern PDAs have the ability to connect to the Internet. A PDA has an electronic visual display, letting it include a web browser. Most models also have audio capabilities, allowing usage as a portable media player, and also enabling most of them to be used as telephones. Most PDAs can access the Internet, intranets or extranets via Wi-Fi or Wireless WANs. Sometimes, instead of buttons, PDAs employ touchscreen technology. The technology industry has recently recycled the term personal digital assistance. The term is more commonly used for software that identifies a user's voice to reply to the queries.

I think the great thing about a PDA is in the name itself - it's a personal digital assistant - it's entire purpose for being is to assist you. It's not there to track you, it's not there to distract you, it's not there to overwhelm you - it just does one simple task as a time - and it does it well.

I think what we need is a modern-day open-source Linux variant of the PDA concept, with modern ideas and modern hardware. Something that sits halfway between a phone and a "smart" device (such as a PineTime). Something that is designed to sync with the Linux operating system and provide a simplified but coherent user experience.

Proposal

So now I will propose such a device... I'll start with some hardware, then talking about the operating system and finally talk about the applications. The idea is to make a realistic proposal that can be achieved with existing hardware and software (for the most part).

Hardware

Ideally I would like to work with Pine on this project, if nothing else because they have a good open-source ethos. A likely candidate base for a Linux PDA would be a PineCube, with the following specifications (which I'll also add notes to):

  • Allwinner S3 Cortex-A7 - This processor is just about fast enough to run a single application at a time.
  • 128MB DDR3 - One would imagine this would allow a running application to safely use 64MB of RAM.
  • 128Mb SPI Flash - This could be just about viable for having a read-only (for the most part) Linux operating system.
  • mSD slot - This would then be used for data and could be used for transferring files/applications too and from the device, including loading updates, etc.
  • 5 MPx OV5640 sensor - There has been a great reverse engineering/driver push through the PinePhone community.
  • 10/100M Ethernet - I imagine this to be largely redundant for a PDA.
  • 802.11 b/g/n WiFi - This will be exceptionally useful.
  • Bluetooth 4.1 - This could be great for syncing and/or connecting to other devices, such as a BT keyboard, PineTime or headphones.
  • USB 2.0 - Additional option for connecting devices.
  • 26 pin GPIO header - I imagine this would be largely used for connecting the touchscreen LCD.
  • Microphone - Communications and/or voice recordings.
  • Passive Power Over Ethernet (PoE) - I imagine this would be largely redundant for a PDA.
  • Interchangeable M12 lens type (wide, fisheye, zoom, etc.,) - Again, likely largely redundant for a PDA.
  • IR LEDs for night vision - Could be useful for taking pictures in low-light conditions.
  • Optional 4.5″ RGB LCD screen - This would of course be exceptionally useful.
  • Battery support - If we want to really use this device, this will be entirely necessary.

I believe the PineCube offers the minimum of features that such a project may want, and therefore would make a viable base platform. Given this dev-kit ships for $30 USD, one can imagine being able to deliver a complete PDA device for $60 - $80 USD (before shipping and tax) and still make a comfortable margin.

Operating System

As already discussed, the idea would be to use Linux. There has already been some effort to mainline into the kernel, with a viable distribution likely being Armbian. As the networking appears to be supported, it should be possible to do initial development work via SSH, although a serial terminal is also supported.

The idea for the window manager will be to recycle the oakwm project ideas and build an even lighter user interface from scratch. Many of the original features from wm2, specifically the window decoration, I found to actually be extremely CPU intensive. As it happens though, for the most part, we don't actually need or want window decoration anyway. Rather than hack wm2 to do what we need, it will likely be easier to simply build a minimal user interface from the ground up.

Oakwm window manager

Oakwm window manager

Essentially the user interface will be approximately like the following:

+--------------------------------+
| [=] [app name]       [v] [|||] |
+--------------------------------+
|                                |
|                                |
|                                |
|                                |
|         Application            |
|                                |
|                                |
|                                |
|                                |
|                                |
+--------------------------------+

A space is reserved at the top for a toolbar that is always accessible and on-top, with the application always being 'full-screen' below. The reason for keeping the toolbar always visible is because it will always be running and be the only way to escape some running application and back into the main menu. Only applications in the 'main window' will get some decent processing time, everything else is either closed or opened periodically.

There would also be an on-screen keyboard for touchscreens, so applications would need to be able to handle dynamic resizing:

+--------------------------------+
| [=] [app name]       [v] [|||] |
+--------------------------------+
|                                |
|                                |
|         Application            |
|                                |
|                                |
+--------------------------------+
|                                |
|           Keyboard             |
|                                |
|                                |
+--------------------------------+

Applications

So it's very likely that applications will need to run in under 64MB of RAM when running with a GUI, so they need to be designed really well. This does have benefits for other devices though, for example, the PinePhone which is also very RAM limited. It also couldn't hurt for modern development to be much more resource friendly!

One thing that I am considering is whether to use the JVM on the PDA. Many older Android phones for example would run a very lightweight JVM for all applications, and these were surprisingly fast all things given.

EDIT: This server now operates on 128MB of RAM, including running nginx, a jvm, pandoc and ffmpeg - 128MB should be plenty if handled correctly.

Whilst C/C++ would be both faster and smaller, the JVM offers instant cross-platform capability and the greatest possibility of multiple device support. The benefit of this would be that you could run exactly the same software on your PDA as you do on Linux, Windows and MacOS - with the small RAM footprint! No Linux lock-in.

The greatest consideration would be whether the resources are available. In theory, running applications whilst the JVM is already started should be quite fast. You can even limit the RAM available considerably in order to prevent OOM and swapping.

The last real consideration is the possibility of some kind of 'app-store'. This would just be a collection of PDA specific friendly apps that are able to be accessed from the app menu. Ideally apps that are to be used with the device would need to have several features, such as low-RAM footprints, be touch-input friendly, be able to handle kill signals elegantly, etc. There would be no reason for example to have GIMP in the app store, as there is not a hope in hell of that running on a PDA (nor would it be the purpose or place of a PDA).

Configuration

This would be the entire device's configuration, such as system theme, language, time/date, internet connection, device switches, etc. Everything that can be adjusted for the PDA and is not application specific would likely find its way in the configuration application.

As there will be so many settings, some form of menu navigation and grouping will be required. It would then make sense to force a reload of the PDA (warm-reset) in order to ensure the entire system state is reset.

Calculator

A very simple initial application is a calculator - if it can't do this then we really are screwed. Unlike some applications though, as we're running Linux, it should even be possible to make use of a solver and really add some value. You could expect it to answer something like $ 2 x = 6 $ for example.

Camera

As the PineCube has a camera (it's supposed to be an IP camera after all), it would make sense to make use of it. This would include photos and videos, although it is quite likely this functionality would largely depend on the write speed of the external micro-SD.

Notes

This is a basic expectation of a PDA device - to be able to store notes. This will mean not only a text viewer, but a text editor. One should expect that it is also RAM efficient and capable of opening large files, as well as searching for text in the large files.

My initial thinking is that it would make use of markdown for markup of the text, meaning there is some very lightweight rich text, whilst still being very compatible with other formats. With the use of pandoc it could easily be exported to anything the user needs (although this would likely never be run on the PDA itself).

Calendar

This is notoriously hard to implement, I would most definitely be looking to use a library for this. Time and dates for example are on their own are insanely hard to correctly implement.

The idea essentially would just be to have something that can view an existing calendar - the capability would definitely be quite limited. It's not entirely clear how everything will be synced across multiple applications as so many different formats are used for email calendars.

Email

Again this is something I would want to use a library for. I imagine the easiest way to handle this would be to sync with POP, IMAP or SMTP. This would require the use of several tricks in order to use very little RAM - the shear amount of data alone could make it near impossible to display for example.

Chat

This would be text-based communication, initially a lightweight IRC client (as state doesn't need to be tracked). Other chat clients would have some expectation that they are active in the background and require some database to store messages. If we remove the need for logging and just keep some messages in RAM (and of course channels to connect to would be a configuration saved to disk), then in theory this would be extremely lightweight.

Browser

This will have to be thought about carefully - the PDA isn't going to be able to use something like Firefox. But a possibility could be Dillo or Netsurf, two 'reasonable' options. If we can use just their libraries, it could be possible to build a very thin single-page wrapper.

Internet Phone

Projects like the ESP32-based WiPhone have shown that it's possible to use a little processing power and WiFi to have an internet connected phone, capable of doing calls. This means that it's quite hopeful that an even more powerful machine is capable of handling phone calls quite easily, likely being also able to do much more - such as audio compression, noise reduction and recovery.

Feed Reader / Podcasts

Building a new RAM friendly RSS feed reader has been on my mind for a while now, I even wrote a blog about it. I already found that Thunderbird was ill suited to the task, as is Akregator. A new RSS feed reader that is speed conscious will be a welcomed new piece of software for the open source community.

Media Player

Given that it can play stuff, why not get it to play music? Perhaps even some small h264 encoded videos? I don't see why this device can't be a very nice little MP3/MP4 player replacement. With BT support, in theory any wireless headphones should be able to play music at acceptable quality.

It could in theory be possible to also watch Youtube videos and the like, possibly through a dedicated wrapper application. This would be something to look at a long-time down the line though.

Maps / Navigation

After using offline maps in FitoTrack as part of my exercise tracking, I have been convinced that offline maps are incredibly useful. Not so long ago I went out for a long walk with a friend late at night, and we ended up using a classical paper map in a petrol garage to find home again. Even if the PDA does not have GPS or any way of locating your position, the ability to just view a map is incredibly useful.

One could imagine a scenario where you are navigating in a town or out in the wilderness with your low-power PDA device, using landmarks to triangulate your position (like they did in the old days).

Given that entire maps of Counties are just a few GBs, it should be quite possible to store this data on an external SD card. Some smart search algorithm could then be used to pull out the relevant data for a given location and then render it on the display.

Future

I am currently in talks in IRC about the PineCube being used for such a purpose. The main points will be in finding out the following:

  1. The current state of the LCD driver. Writing an LCD driver for the Linux kernel from scratch is not something I've done before, so it will be a lot of pain if I end up having to do that.
  2. The current state of the PineCube project overall and ensuring they are still in stock.