The search for the holy grail of embedded operating systems

Imagine: You get a fantastic idea to create an 'intelligent' toaster. You've figured out how to use AI to make the perfect toast! You get excited and onboard a bunch of smart people to work on the AI software stack. As there are plenty of OEMs to design a high-end toaster, you are not too concerned about the toaster's mechanical parts. But hey, what about the brains of the device?

You'll have to run an operating system (OS) to control your AI inferencing models and connect with your back-end systems. But you realize that your end customers would not care about the platform choice. What matters is that it is cheap to create and maintain and easy to develop .

Surely, there should be a platform that meets these requirements, right? Well, sort of, we are in 2022, after all. While no one unicorn embedded OS does it all, there are a few obvious choices for various experience levels and scales. Let us understand these choices and figure out how to pick one.

The non-contenders

Let us start with the platform choices that can be honorable mentions but are not exactly worth considering:

1. Android

The Android open-source project kicked off ten years ago by touting itself as a general-purpose operating system. Since then, Google has gradually undermined that promise as it continues to lock down the ecosystem to improve the mobile phone user experience. Various failed spin-off attempts like "Android Things" have tried to get back into the general embedded systems space. But as of now, Android is not conducive for general-purpose embedded devices and should be avoided.

2. FreeBSD

The reason people pick FreeBSD is to avoid the GPL license of the Linux kernel. A good example is Sony with their PlayStation OS. FreeBSD also has a mature network stack and can be found in network hardware like enterprise firewall devices. Overall, FreeBSD has its niche, but it is not a recommended general-purpose option.

3. Google's Fuchsia

Fuchsia made headlines when Google announced that they had shipped the first production system (1st gen Nest Hub) with the Fuchsia OS. It was developed from scratch as a modern alternative to Linux. It is not ready for general purpose use yet but could be something to look out for in the future.

The contenders

Okay, now we are talking. I would broadly categorize the viable options into the following groups:

1. Custom distros: Yocto/Buildroot
2. Open distros: Debian on ARM/Ubuntu ARM server
3. General-purpose distros with vendor support: Timesys Factory/Wind River Linux
4. Proprietary distros with closed ecosystems: Ubuntu core/Windows 10 IoT Core/BelenaOS

In general, I visualize these options in the following linear spectrum:

There is a clear correlation between customizability (hackability) and the effort required to develop and maintain the OS. On one extreme, you have an option like Yocto, which provides maximum customization flexibility but leaves you to manage the associated complexity. You need to invest a lot of effort to validate and maintain all this. On the other side of the spectrum, you have opinionated and rigid ecosystems like Ubuntu Core and BalenaOS, where someone else manages the complexity for you.

Let us look at these options in more detail:

1. Custom distros

I would group open-source options like Yocto and Buildroot under this category. These options are not an OS distribution but a framework for building a custom distro. You can pick your choice of toolchains, libraries, features, etc. The caveat is that everything is on you. It includes setting things up, functional/stability validations, patching security CVEs, etc. You will not get any pre-built binary packages for the apps and libraries, so be prepared to build everything from the source. The bright side to this is that service providers offer off-the-shelve solutions to common problems.

A good example is Mender.io which provides paid OTA support via a Yocto layer. There are numerous open-source Yocto layers that you can leverage to add features like secure boot support, recovery boot environment/factory reset support, A/B filesystem support, etc. Overall, this is a go-to option for experienced teams.

2. Open distros

This covers popular open-source distributions like "Ubuntu ARM Server" and "Debian on ARM". It's the best option for rapid prototyping as you can get things done quickly, with minimum effort. Though your customizability is limited to what the distro supports, it is typically good enough. The biggest boon here is that you can just "apt install/upgrade" popular apps and libraries. While you can take these options into production, you'll have to fight the system to make extensive customizations (like forking the kernel). On a side note, userspace conflicts are no longer the showstopper (like they used to be) thanks to application containerization technologies, like Docker and Podman.

3. General purpose distros with vendor support

Many large service companies maintain and offer paid support for proprietary distros and build systems. Some of the big names are:

• Timesys
• Seimens
• Wind River Systems
• LynuxWorks

Every vendor runs Linux under the hood, but you get support and validation. Their offerings cover a large spectrum, from Yocto's flexibility to opinionated pre-packaged distros. The common factor is that they customize the solutions to your needs and usually offer security updates and back-end infrastructure for OTA updates. While they are not inexpensive, I would still recommend this route for upstarts that do not have it or need in-house platform expertise.

4. Proprietary distros with closed ecosystems

Finally, we have fully managed operating systems that provide a streamlined workflow to abstract away the low-level nitty-gritty. The popular options are:

• BalenaOS
  This is an operating system optimized to run containers. All your customizations must be containerized, allowing the platform to be managed independently. Besides the additional overhead of the container runtime and the hoops to jump through to access and share hardware, this has a compelling proposition for people targeting more modern hardware platforms.
• Ubuntu Core
  It is similar to BalenaOS in concept but uses Ubuntu's "Snap" application package format to sandbox apps. I must say that I am not a huge fan of Snaps. It is hard to develop and get seemingly trivial stuff up. For example, an X-server instance is quite an ordeal. But I expect things to improve, especially for GUI apps, with "Ubuntu Frame" coming in. Having said that, this is not an option I would recommend currently.
• Windows 10 IoT Core
  You can leverage some cool synergies if you are heavily invested in the Microsoft ecosystem. The biggest drawback is that you must partner with Microsoft-approved OEM for customizations. But if you already have connections with Microsoft OEM partners or are targeting x86 platforms, this option may offer a good value proposition.

Each of these proprietary distro options are highly opinionated. They asks us to invest in them and buy into what is effectively a walled garden ecosystem. So, there is absolutely no way to switch between platforms easily. This is the primary reason I must advise against these options.

Conclusion

Long story short, no one "holy grail" embedded operating system will work for everyone. The choice of the operating system will depend on your expertise and particular niches. I would summarize the decision tree as:

• For prototyping: Ubuntu
• When you want to be in the driver's seat (for long term support, etc.): Yocto
• When you do not need complete control: Timesys

Need help in finding the platform suited to your requirements? Get in touch with our experts. Today!