Introduction

The SmolPhone project aims to build a mobile phone with limited hardware that still brings value to its users, and enables them to easily create and edit applications using a scripting environment.

The project provides libraries and drivers, and currently targets the SmolPhone Device, which was developed by our team. The project also targets the PicoCalc with limited functionalities. These two devices use a Raspberry Pi microcontroller with limited memory (264 to 520 KB).

This book provides a high-level overview of the project and helps you to get started. If you have any question, you can reach us via our Discord server.

Getting Started

The SmolPhone repository is where you can find the SmolPhone source code and open issues.

This is a monorepo that includes several subprojects:

• os/ contains the SmolPhone libraries and drivers for the devices we support. It also contains some examples that can be run on your machine.
• device/ contains demonstrations that can be compiled then flashed on the SmolPhone Device.
• picocalc/ contains demonstrations that can be compiled then flashed on the PicoCalc
• docs/ contains the markdown book you are currently reading.
• assets/ contains ressources used by the demonstrations.

os/, device/, and picocalc/ are Cargo workspaces. They are kept separate because they don’t target the same hardware. Cargo doesn’t yet support crates targeting different hardware within the same workspace.

Clone the repository

First, clone the repository:

git clone git@gitlab.inria.fr:smolphone/smolphone.git

If you don’t have an account on the forge, use the HTTPS URI instead:

git clone https://gitlab.inria.fr/smolphone/smolphone.git

Install the toolchain

We use rustup for installing the Rust toolchain. Execute the following command in each Cargo workspaces to install the required toolchain.

rustup toolchain install

Install a C compiler for the arm-none-eabi architecture. It is required by the os/smolphone_wren/build.rs script. For example, you can install arm-none-eabi-gcc using your system package manager. Some versions of arm-none-eabi-gcc don’t include header files such as stdlib.h and string.h. To add them, you can install newlib (arm-none-eabi-newlib or newlib-arm-none-eabi).

Execute an example

The os/ workspace contains some demonstrations of the SmolPhone OS that can be run on your desktop computer. These examples are linked dynamically to SDL 2.0.26 or newer. For example, you can try our dictionary demo:

cd os/
cargo run --release --bin dictionary

List the available demos using the following command:

cargo run --bin

These binaries are defined in the smolphone_examples crate.

Contributing

Thank you for investing your time in contributing to this project. There are many ways to contribute and we value all of them.

Posting or commenting an issue

If you find a bug or want to discuss a feature, open an issue.

When posting a new comment on an issue, make sure your comment adds value. Don’t post a comment just to get attention.

Remember that we are humans: we like support, and we expect kindness :)

Writing code

Any contribution should resolve a work item. If there is no work item about the change you want to introduce, then create one for discussing that.

We try to follow most of the Rust API guidelines and the Rust standard library policies.

Fork or clone the repository

If you have a write access to the SmolPhone repository, you can simply clone it:

git clone git@gitlab.inria.fr:smolphone/smolphone.git

Otherwise, fork the repository, and then git clone your fork.

Create a dedicated branch for each feature or bug that you are working on:

git switch -c my-new-branch

Open the project

The project consists of three Cargo workspaces: os/, device/ and picocalc. If you are using Visual Studio Code, open the smolphone.code-workspace file using the command Open workspace from file. This will provide you a VSCode workspace that contains these subprojects. For other editors, open each Cargo workspace in a dedicated editor instance to take advantage of the rust-analyzer LSP. The cargo workspaces are kept separate, because Cargo doesn’t support setting distinct targets within the same workspace yet.

Install the toolchain

See the “Install the toolchain” section of the Getting Started guide.

Format your changes

Execute cargo fmt to format your code. You can also use our cargo alias: cargo f.

Lint your changes

Execute cargo clippy --all-targets --all-features to lint your code. You can also use our cargo alias: cargo l.

Also check the docs using cargo doc --all-features --no-deps, or our cargo alias cargo d.

Build and test your changes

Execute cargo test --all-features to execute unit tests and doc-tests. You can also use our cargo alias: cargo t.

The device/ and picocalc/ workspaces don’t support tests because of the hardware they target.

Commit your changes

This project follows a specific format for commit messages. A commit head starts with an optional scope between square brackets and ends with the capitalized title. The scope corresponds to the affected crate. For brevity, we omit the smolphone_ prefix from the crate name. The commit body details the change.

For example, the following commit affects the os/smolphone_graphics crate:

[graphics] Improve image and font abstraction

- A font now uses a binary image internally.
- A glyph is now a subimage.
- Add a method to get the bounding box of a subimage.
- Add GraphicsContext that stores fonts and images used by the
  drawing phase.

Please rebase your change on main before creating your merge-request. We maintain a linear Git history.

Code philosophy

Rust version

The SmolPhone project uses Rust 1.88 edition 2024. We only update when dependencies move to a more recent version or a feature is required.

Each Rust workspace contains a rust-toolchain.toml file that defines the Rust version. It must be in sync with the rust-version set in the workspace’s Cargo.toml.

Minimize dependencies

This project embraces a strict policy regarding dependency management. This aims to avoid potential security vulnerabilities and software bloat.

1. Think twice before adding a new dependency.

   If you think a dependency should be added, then justify why. This justification should be written in the commit that adds the dependency.

2. Avoid crates with dozens or hundreds of direct or indirect third-party dependencies.

   Third-party dependencies are crates that aren’t authored by the same organization or author.

   You can see the tree of dependencies of your project using cargo tree.

3. Audit small and untrusted crates

   Sometimes there is no better choice that a package authored by a single developer. In this case you should take the time to review the source code.

   Audit every update of the package. You should depend on an exact version of the package.

   Note that this doesn’t apply the restriction on the dependencies of untrusted-package. This is why you should avoid untrusted packages with untrusted dependencies.

We trust the following organizations:

• rust-lang The organization maintains:
  • the Rust toolchain (cargo, rustc, rustup, …)
  • rust-bindgen
• rust-embedded The organization maintains:
  • cortex-m
  • embedded-hal
  • embedded-alloc
  • embedded-io
  • heapless
• rust-embedded-community The organization maintains:
  • embedded-sdmmc-rs
• knurling-rs The organization maintains:
  • defmt
  • flip-link
  • probe-rs
• serde

When you add a new dependency, you should also ensure that its license is compatible with this project. You should also review the licenses of its direct and indirect dependencies. You can list the license of all dependencies using cargo license.

License

The Rust community double licenses its libraries with the MIT and Apache 2.0 licenses. This double licensing provides flexibility for software that depend on them.

We chose to diverge from this practice in order to protect our work. All of our libraries and software are licensed under the Mozilla Public License 2.0 (MPLv2). This license requires sharing any modifications of our libraries and software, but still allows to freely depend on them without any copy-left contamination.

Thus, a software or library that depends on ours can still be licensed under a permissive license such as MIT or Apache 2.0. This makes ours libraries reusable by the Rust community.

Design goals

The SmolPhone project aims to build a mobile phone with limited hardware that still brings value to its users, and enables them to easily create and edit applications using a scripting environment. It targets the following hardware:

• High-end microcontrollers such as the RP2040, which is a 32-bit, dual-core ARM Cortex-M0+ processor running at 133 MHz with 264 KB of SRAM.
• Several megabytes of readonly (non-volatile) memory.
• Memory-in-pixel reflective LCDs with a slow data transfer that supports partial refresh, such as the Sharp LS027B7DH01 which is a $400 times 240$ monochrome display with a serial connection.

Thus, the phone’s memory is more limited than its processing power. Notably, we have far less mutable memory than readonly memory. Also, the display should be partially refreshed as much as possible to reduce the perception of slow data transfer between the memory and the screen.

The software platform that we want to design must meet several quality criteria:

• C1.a: Low memory usage
• C1.b: Efficient use of readonly and mutable memory
• C2.a: Familiar and arguable architecture and abstractions
• C2.b: Readable, documented and well-tested code
• C3.a: Provide enough functionality to implement usual phone applications
• C3.b: Provide a scriptable environment similar to HyperCard or its modern variant Decker

Crates

The Figure 1 shows a simplified view of the dependency graph between the crates of the project. The smolphone/os workspace contains most of the crates that are libraries, drivers, and demos. At the base of the graph are the libraries:

• smolphone_dict provides efficient dictionary lookup and parsing for the DICT format.
• smolphone_fs offers a file system abstraction and implements it for the Rust’s std::fs module and embedded-sdmmc crate.
• smolphne_graphics provides a framework for creating and rendering a graphical user interface.
• smolphone_wren is a safe wrapper around the Wren virtual machine. Wren is a scripting language used to write applications’ logics.
• smolphone_wren_modules exposes some SmolPhone libraries as Wren modules that can be accessed in a Wren script.

The smolphone_sharp_memory_lcd and smolphone_tca8418_keypad crates are divers for the display and the keypad used by the SmolPhone Device. smolphone_apps provides applications such as a dictionary application. It defines the application logics and the user interface. smolphone_examples and the crates of the smolphone/device workspace depends on the smolphone_apps crate and set up the runtime. smolphone_examples relies on the embedded-graphics simulator which in turn relies on SDL.

SmolPhone Device

At a glance the SmolPhone Device consists of a display, a keypad, a board with a microcontroller, and a LTE-M modem.

The display is a Sharp memory LCD LS027B7DH01. This is a 400x240 monochrome reflexive LCD with very low energy consumption. The keypad is a TCA8418 50-key matrix. The board is a PICO2-XXL. It features a RP2350B which includes two 150 MHz ARM Cortex-M33, 520 kB of memory, 16 MB of flash storage, and a micro SD-card slot. The LTE-M modem is a nRF9151 from Nordic Semiconductor. The modem provides a 64 MHz ARM Cortex-M33 processor, 256 kB of memory, and 1 MB of flash storage for the user.

The Figure 1 provides a high-level overview of how these components are connected to each other. The Figure 2 shows a photo of the current device.

Modem

The modem consists of two cores: the cellular core and the application core. The cellular core contains the firmware and is responsible for handling cellular functions. The application core can be programmed.

To add cellular capabilities to the SmolPhone device, we explored two architectures:

1. The serial modem architecture
2. The custom API architecture

In the serial modem architecture, the board sends directly AT commands to the modem. The application flashed on the modem is the Serial Modem add-on. This application forwards the AT commands to the modem and provides different operating modes. For example, it handles AT commands and data multiplexing.

In the custom API architecture, we program a modem application that handles the low-level details such as opening sockets and sending HTTP requests. The board communicates with the modem application though a custom API that we designed.

We chose the second architecture because it allows taking full advantage of the application core. This avoids an overhead in parsing and printing AT commands. During our testing, we encountered several bugs with the serial modem add-on. The add-on doesn’t appear to be intended for production use.

Flashing

The flashing process allows copying a program on the SmolPhone device. A program can be flashed either via a debugging probe or via the micro USB port of the Raspberry Pico. Make sure you don’t confuse the USB-C port, which provides power to all peripherals, with the micro USB port on the Raspberry Pico.

Flashing with a debugging probe

Connect the debugging probe to the Arm Serial Wire Debug (SWD) of the Raspberry Pico. Connect the main power via the USB-C port. Press the power button just before flashing the device. This provides power to peripherals like the display, the SD card, and the keyboard. If you power these peripherals too soon, you can end up in a state where the keyboard is frozen.

You can flash the program using cargo embed or the VSCode debugger.

Using cargo embed

First install probe-rs. cargo embed is installed within probe-rs.

You can check that the probe is correctly connected using the probe-rs list command.

Then run cargo-embed with the desired program. List the available programs running the cargo run --bin command.

cargo embed --release --bin dictionary

This compiles the project, and you should get the following output:

Profile default
 Target target/thumbv6m-none-eabi/release/dictionary

Then press the power button to power up the device and flash the device. Note that if you take too much time to power up the device, the flashing process will abort.

defmt messages should be logged on the console.

Debugging with GDB

Install gdb-multiarch or arm-none-eabi-gdb depending on your Linux distribution.

In the Embed.toml file, set default.gdb.enable to true. Flash the program without the --release option:

cargo embed --bin dictionary

A TCP port is opened for the GDB server. To connect the GDB server, run the following command:

<gdb-program> -q -x embed.gdb <program-file>

<gdb-program> is either gdb-multiarch or arm-none-eabi-gdb. <program-file> corresponds to the program binary. For example, target/thumbv8m.main-none-eabihf/debug/dictionary or target/thumbv6m-none-eabi/debug/dictionary depending on the microcontroller architecture.

See the following resources for more details:

• Tooling page of the embedded Rust book
• debugonomicon

Using the VSCode debugger

First install probe-rs and the probe-rs VSCode extension.

We provide a debug launcher (./.vscode/launch.json) that automatically launch probe-rs. You probably want to edit the programBinary property to point to the program you want to debug.

Flashing with the micro USB port of the Raspberry Pico

We provide a custom runner script (.cargo/runner) that is automatically executed on cargo run. The script must be executable. The runner uses picotool to flash the device. Install it from your distribution repositories.

Before connecting the Raspberry Pico to your computer, hold the reset button. It is a little white button on the Raspberry Pico.

Then run cargo run --release to flash the device.

cargo run --release --bin rphal_display

Developing applications

For now, the application and the OS are basically the same thing. An application is flashed directly onto the device, and some parts of the application logic can be customized by editing the scripts.

An application is usually split in two parts: the application logic and the device-specific part. This allows us to share the application logic across several devices, such as a desktop computer and a SmolPhone device.

The application logic is located in the os/smolphone_apps crate. The device-specific code is located in a _examples crate:

• The os/smolphone_examples crate contains applications that can be run on a desktop computer.
• The device/ workspace contains applications for the SmolPhone device.
• The picocalc/picocalc_examples crate contains applications for the PicoCalc device.

An application consists of a deck of cards. Each card describes an application screen. A card is a tree of widgets and is split into a configuration and a state. The configuration doesn’t change and is part of the program binary. Conversely, the state can change during the execution of the program. It is stored in the volatile memory. The widget layout is part of the configuration.

High-level architecture

In this section, we will explore the similarities and differences between The SmolPhone and other frameworks. First, we examine the graphics pipeline, which is responsible for converting a widget tree into pixels on the screen. Next, we describe how we handle user inputs and change the state of the user interface. Finally, we describe application scripting.

Graphics Pipeline

The SmolPhone uses a commonly-used graphics pipeline to turn a widget tree into pixels on the screen. The pipeline consists of three main phases depicted in the Figure 1:

The layout phase computes the size and the position of each widget on the screen. A size and a position form a bounding box. To lay out the widgets, the tree is traversed twice: first in a bottom-up fashion and then in a top-down fashion. The first traversal computes the minimum size of each widget based on the minimum sizes of its children. The second traversal distributes the remaining space of the widgets to their children and computes the positions of the widgets relatively to their parents. Flutter and SwiftUI use dedicated widgets to express layout constraints. Conversely, we attach layout constraints to each widget. This avoids adding lots of widgets just for layout purposes. This also has the advantage that layout constraints can be removed if the bounding boxes can be determined during compilation.

The render phase generates a list of graphics primitives, commonly named the display list, from the widgets. Graphics primitives are rectangles, images, and glyphs. This phase is responsible for converting texts into glyphs and uses a theme to render the widgets. Modern frameworks enable widgets to cover multiple layers simultaneously, allowing them to partially overlap their children. Also, a widget can sometimes exceed the bounding box of its parent. We chose a tiling paradigm in which every widget is contained within the bounding box of its parent, doesn’t overlap its siblings, and is always rendered above its parent. It is claimed that this paradigm is simpler to implement by the Oberon project. Generating the display list requires only a preorder traversal of the widget tree.

The draw phase processes the display list and draws its graphics primitives on the screen. Modern graphics pipelines usually use two frames buffers: one to draw the graphics primitives and one to copy the frame to the display. The SmolPhone does not have enough volatile memory to afford a frame buffer. To meet C1a, we use the scanline rendering algorithm to draw the screen line-by-line. A single line buffer is required. However, we would like to explore multi-line buffer in the future if it makes drawing more efficient.

Some graphics pipelines can have more phases. For example, Flutter adds a compositing phase between the render and draw phases to merge the display list of several rendered layers into a single one.

The Figure 2, Figure 3 and 4 illustrate the graphics pipeline with a to-do list application. The Figure 2 shows the widget tree of the application. It consists of several widgets: three containers, a text field, a button, a text label, a dynamic container, two checkboxes and two texts. The Figure 3 in green the computed bounding boxes of the widgets during the layout phase. Some bounding boxes are omitted in this figure for sake of simplicity (the dynamic container and the two containers enclosing each to-do list item). Finally, the Figure 4 depicts the drawn screen.

Currently, the layout phase doesn’t have access to the widgets’ state. This means that a widget with a dynamic content, such as a text widget, cannot be sized based on its content.

The graphics pipeline typically operates in either immediate or retained mode. In immediate mode, the entire pipeline is executed to display each frame. In retained mode, the pipeline is executed entirely to display the first frame, and then only partially for subsequent ones. Most of modern frameworks retain the widget layout tree and re-layout it when needed. They can also retain a part of the display list to efficiently handle screen scrolling. They also save drawing result as textures to speed up the draw phase.

The SmolPhone uses a minimal retained mode that retains the widget layout tree. Only widgets whose state has been updated since the last displayed frame need to be re-rendered. In the previous example, if a user toggles a checkbox, then only this checkbox has to be rendered and then drawn on the screen. In the future, we would like to explore the possibility of retaining part of the display list to efficiently handle screen scrolling. We cannot texturize the drawing results because it could contradict C1a.

Layout Phase

The layout phase is responsible for computing the size and the positions of the widgets. This computation is based on the layout constraints of the widgets.

Every widget has an associated layout constraint. A layout constraint defines:

• The minimum size in pixels of the widget.
• The padding in pixels that separates the widget boundaries from the contained widgets.
• The gap in pixels that separates adjacent contained widgets.
• The arrangement of contained widgets: either horizontal or vertical.
• The width and height eighths that determine how much of the remaining space the widget will take.

The Figure 1 shows an example of layout constraints. In this example, the size of the widget fits its content. The contained widgets are arranged horizontally.

The layout phase consists in two steps. The first step computes the minimum sizes of every widget. It traverses the widget tree in a bottom-up fashion. The second step distributes the remaining space to the widgets and computes their absolute positions. It traverses the widget tree in a top-down fashion.

The minimum size of a widget corresponds to the component-wise maximum between its fitting size and the minimum size defined in its layout constraint. The fitting size corresponds to the size required to fit the content. In the Figure 1, the fitting width of the widget is equal to the sum of the widths of its two contained widgets, the gap, and twice the padding. The fitting height is equal to the maximum between the heights of its contained widgets plus twice the padding.

If a widget has nonzero width and height eighths, then it takes some of the remaining space of its parent widget. For a horizontal arrangement, the remaining width is the difference between the parent width and the parent fitting width. The remaining height is the difference between the parent height and the widget height. The root widget has an implicit parent that is the display.

The absolute positions are computed by simply taking the sizes of the widgets, the padding and gap between them.

Let’s take the widget tree in Figure 3. To simplify their designations, all widgets are indexed from 0 to 5. Their layout constraints are enclosed between parentheses. The Figure 3 shows the computed sizes and positions for a display size of 160x160.

└─ [0] Container (vertical, padding: 8, gap: 8, widthEighth: 8, heightEighth: 8)
   │
   ├─ [1] Container (horizontal, gap: 8, widthEighth: 8)
   │  │
   │  ├─ [2] TextField (minSize: 56x0, widthEighth: 4, heightEighth: 8)
   │  │
   │  └─ [3] Button (padding: 4)
   │     │
   │     └─ [4] Label "Search" (minSize: 48x8)
   │
   └─ [5] Container (widthEighth: 8, heightEighth: 8)

State of the Art

Cursor-Based Layout

In cursor-based approach, a widget is drawn at the current cursor position, then the cursor is moved to a new position.

The main disadvantage of this approach is that you cannot easily create responsive user interfaces. You have to manually compute the positions of the widgets. It also inherently imperative that prevents to perform some optimizations: It is not possible to easily render only the widgets that have changed and those that intersect the same lines. This doesn’t suit well retained modes.

This approach is often used by immediate mode libraries to avoid a layout phase entirely.

Windows Presentation Foundation (WPF)

WPF uses a two-pass approach: a measure pass and an “arrange” pass. The first pass computes the sizes of the widgets passing down the available space. The second pass computes the final sizes and the widget positions.

Flutter

Flutter lays out widgets by recursively traversing the tree. Each widget receives its size constraints from its parent, determines its size and the position of its children.

Size constraints are thus propagated from top to bottom. A widget derives the size constraints of each child from its own size constraints. The maximum size of the root corresponds to the available space. Size constraints are strict: each widget has to choose a size that fits its constraints. This is a key design decision that avoids backtracking and thus ensures a linear complexity. One consequence of this decision is that a widget can impose a size to a child. For example, a checkbox can be forced to take a larger size than its intrinsic size. In such cases, the widgets are often rendered using their intrinsic sizes and is padded with the remaining space. In practice, Flutter can backtrack when it lays out text. However, this is isolated to specific parts of the widget tree.

Once its children have computed their size, a widget computes its own size and places its children. Flutter uses relative positions. Thus, it doesn’t need to know the position of a widget to place its children. This also avoids recomputing the layout when scrolling: only an offset is updated. Also, Flutter has a concept of layout boundaries which allows isolating layout re-computations.

Flutter allows distributing the remaining space among the flexible children. The remaining space is determined by first visiting non-flexible children. Each flexible child defines a flex factor that determines how much of the remaining space it takes.

Slint

At first glance, Slint uses a system that looks like FLutter: Size constraints go from top to bottom, sizes and positioning go from bottom to top.

However, Slint uses a different architecture. While Flutter uses an explicit layout phase, Slint uses a reactive system. Sizes and positions are lazily computed and re-computed. Also, Slint leverages its compiler to perform some layout work at compilation time.

Web

The web uses a very complex layout system that use backtracking. Thus, the layout phase is usually not linear.

Constraint Solvers

More complex layout engines allow expressing rich constraints between widgets. So, they use solvers such as SMT solvers to lay out the widgets.

Draw Phase

Draw commands defines where and what to draw on the screen. The SmolPhone supports the following draw commands:

• Border: transparent rectangle with solid borders.
• Fill: rectangle filled with a given color and without borders.
• FillCircle: circle filled with a given color and without borders.
• BinaryImage: an image.
• Text: a line of text without breaking.

A draw command has a bounding box that determines where to draw on the screen.

In contrast to most render phase, the SmolPhone delays the rendering of text into glyphs to the draw phase. This saves memory by avoiding the insertion of each glyph into the display list. This comes at the cost of redundant computations, because the glyphs of a line of text are rendered for each lines of pixels. For example, if the font height is 16 pixels, then the glyphs of a line of text have to be rendered 16 times.

Sending to the display

The SmolPhone prototype uses the Sharp Memory-in-pixel LCD LS027B7DH01. It is a 400x240 monochrome display. The display is connected to the microcontroller board via the Serial Peripheral Interface (SPI) protocol.

The display supports an SPI clock speed of up to 2 MHz, enabling 20 frames per second. In practice, it can often be overclocked. For example, the Playdate uses the same display and supports from 30 to 50 frames per second. Its SPI connection is thus overclocked to at least 5 MHz.

The display is able to update multiple lines within a single SPI transaction. Each line consists of 400 bits with a prefix of 16 bits that contains the line number. The transaction ends with a 16 bits trailer of dummy data. Thus, to update n lines in a single SPI transaction, we transmit L(n) bits.

\( L(n) = \begin{cases} 0 & \text{if } n = 0\\ 416n + 16 & \text{otherwise} \end{cases} \)

This takes LT(n) seconds and allows sending LPS(n) transactions per second.

\( \text{LT}(n) = \frac{L(n)}{F} \)

\( \text{LPS}(n) = \frac{F}{L(n)} \)

To update the entire display with n-lines transactions (\( 0 < n <= 240 \)), U(n) bits are transmitted.

\( U(n) = \frac{240}{n}L(n) + L(240 \mod n) \)

Given a SPI clock speed of F hertz, the SPI connection is able to transmit F bits per second. The theoretical time to update the entire display is FT(n) and the number of frames per second is FPS}(n).

\( FT(n) = \frac{U(n)}{F} \)

\( \text{FPS}(n) = \frac{F}{U(n)} \)

\( \text{FPS}(n) \) forms a hyperbola with an asymptote at around 20 frames per second, as depicted in Figure 1. See the Figure 2 for the values obtained. To reduce the number of values we only reported the factors of 240 for n.

Figure 3 highlights the importance of updating as few lines as possible. For example, if we update 24 lines in batches of 8 lines, we can perform \( \frac{\text{LPS}(8)}{\frac{24}{8}} \approx 199 \) updates per second.

To measure the time necessary to update the entire screen in real conditions, we created a program that updates every line of the screen on each iteration. Lines are updated in order. Thus, if we update the display by batches of 120 lines, then the first transaction updates the first 120 lines. We found that the first iteration took longer than the others. We also found that excluding this first iteration caused the average to stabilize more quickly, after around five to ten iterations. Thus, we decided to treat the first ten iterations as a warmup, computing the average time taken to update the entire screen over the next twenty iterations.

The transmission of data on the SPI can be managed by the microcontroller or by the Direct Memory Access controller (DMAc). In the Rust ecosystem there exists two main Hardware Abstraction Layers (HAls) for the RP2350: rp-hal and embassy. We implemented our own DMA adapter for the rp-hal. Embassy provides a built-in support or DMA and puts the microcontroller into sleep mode when the DMA is used. So, we tested our program in four different settings, varying the HAL and the method by which the transmission is managed, i.e. by the MCU or the DMA controller.

The Figure 3 shows the obtained times for a SPI clock speed of 2 MHz. The Figure 4 and the Figure 5 show the obtained times for a SPI clock speed of 5 MHz. Embassy is slightly slower than rp-hal. However, the difference narrows as n increases. Using the DMA controller to manage the transmission is slower than using the MCU at 2 MHz. Conversely, it is much quicker at 5 MHz. We found than using the DMAc provides gains above 2.5 MHz.

One advantage of using the DMAc is the ability to execute other tasks while the transmission is in progress.

Handling of user inputs

Some widgets are interactive and react to input events. Frameworks use the widget tree to determine which widgets have been hit by a pointer through a process called hit testing. Frameworks dispatch events directly or propagate them along the tree. In the direct dispatch approach, the framework sends the events to either the innermost interactive widget under the pointer or the focused widget for keyboard events. In the propagation approach the events travels downward from the root to the innermost widget and then travel backward. Flutter and Qt use a hybrid approach that uses a direct dispatch approach, but allow explicit event propagation upward. We use a direct dispatch approach and do not offer any way of escaping this behavior for now. Thus, a widget that reacts to an event will always consume that event.

Frameworks use either a callback-based approach or an event-based approach to handle widget reactions. In a callback-based approach the application and the framework registers callbacks to call when the widget reacts to an event. In an event-based approach widget emits their own events that are dispatched to a centralized handler or to dedicated handlers. Modern frameworks usually use the callback-based approach because it is well suited to declarative user interfaces. The event-based approach allows a better decoupling between the user interface and the application, which improves testability. This is why we choose the event-based approach. However, we haven’t ruled out using a callback-based approach on the user side. A declarative user interface can easily be converted to an event-based approach following a compilation step.

Handling of changes

Based on widgets interactions, an application can change the state of the user interface. In the to-do list application, clicking on the button adds a new item to the to-do list. Legacy frameworks usually use an imperative approach where the application mutates the state directly and manually trigger the graphics pipeline. This style often results in the screen being out of sync with the state. This is why modern declarative frameworks use reactive state management. Reactive systems are either fine-grained or coarse. Coarse reactive system defines the interface as a function over the state. Using some diffing algorithm they are able to compute what part of the interface have changed. Fine-grained reactive system registers dependencies between the state and the affected parts of the interface enabling to identify which parts have to be re-layout, re-rendered, or redrawn. Slint uses the fine-grained approach, while frameworks like React Native, Flutter, and SwiftUI use the coarse approach.

The coarse reactive approach implies generating a new widget tree before diffing it with the previous one. While the use of persistent data structures allows reducing the cost of building a new tree, it still represents some cost in terms of memory allocation and seems not suited to the restriction we have. Conversely, the fine-grained reactive approach requires the implementation of reactive data types that register dependencies using callbacks and use an asynchronous executor to propagate the change among the dependent widgets. Changed widgets are then marked as dirty.

For these reasons we decided to not choose a reactive approach and use a modernized imperative approach. The application mutates the widgets via a layer that registers which widget have been potentially updated. Based on this information we are then able to determine which parts of the screen have to be rendered. Our approach is less fine-grained than Slint’s, so we can render and redraw more parts of the screen than Slint could. In contrast to the fine-grained approach, we save memory by avoiding the registration of callbacks.

Application scripting

We want to make it easy for users to create their own applications, so we allow them to write the application logic as scripts. Currently, we use the #link(“https://berry-lang.github.io/”)[Berry scripting language]. It is an Object Oriented language that claims to require only four kilobytes of RAM.

All the widgets with which the scripts can interact have an associated object in the scripting runtime. These objects are created before interpreting any scripts. The script for the user interface of the @fig-to-do-list-app has access to the text field, the button, the checkboxes, and the item text. The objects provide methods that execute native code to modify and query the state of the widgets. For example, a script can retrieve the content of a text field, change the content of a text, toggle a checkbox, etc.

To make these objects more accessible from the user script, each object associated with a static widget has a name. In the @fig-to-do-list-app, only the text field and the button are static. The checkboxes and item texts are not static because they are dynamically created. To get access to them the dynamic container is named and has an associated object. It allows traversing the list of items.

On the user script side, we use a callback-based approach to setup the widgets’ reactions. For example, the script for user interface of the @fig-to-do-list-app could attach an on-click callback to a button. The host executes this callback when it processes a click event coming from the corresponding button.

For example, the following script attach a callback to the button named ADD_ITEM. Upon click on this button, the callback retrieve the content of the text field named ITEM_TEXT, call a function create_new_item that create a new item and add it to the to-do list.

ADD_ITEM.on_click = def()
  var todo_item = ITEM_TEXT.get_text()
  create_new_item(todo_item)
end

Currently a single script is used to describe the application logic. In the future we plan to explore the idea of attaching a script to every widget that the user can interact with. For example, the following script could be directly attached to the SEARCH button:

def on_click()
  var todo_item = ITEM_TEXT.get_text()
  create_new_item(todo_item)
end

To reduce the cost of instantiating an object for every widget that the script can interact with, we could instantiate objects on-demand. So only widget that the script interacts with will have a corresponding objects in the scripting runtime.

Data representation

Because of our memory quality criteria C1a and C1b, we must pay attention to how data for the widgets and the graphics phase are represented and organized.

Configurations and states

We split every widget into a configuration and a state to meet C1b. The configuration doesn’t change and can be embedded in the program binary. Assuming that the program is not loaded into volatile memory, this saves volatile memory. Conversely, the state can change during the execution of the program. It is stored in the volatile memory.

While some data is always categorized as either a configuration or a state, depending on the use case, some data can be either. For instance, if a user interface is always displayed on screen at a specific size, then the bounding boxes of the widgets can be configurations. Conversely, the bounding boxes of dynamically created widgets are always part of their state.

Data structures

We heavily rely on data structure backed by arrays. This ensures a reduced use of memory and an efficient access and linear traversal.

The structure of the widget tree is implemented with an array where nodes are arranged in a prefix order. Each node indicates the number of nodes in its subtree and its height. This forms a sum tree. Each widget is associated to a node and is identifiable with the node index.

Dynamically created widgets are isolated such as most of the widget tree is static and is thus a configuration. These widgets can only be added to dynamic containers. A dynamic container uses a unique configuration for all its children widgets. This configuration acts as a template.

A widget tree configurations without its state is called a component.

Data layout

Data layout is particularly important in environments with limited memory. For example, combining a one-byte data type with a four-byte data type results in an eight-byte data type due to data alignment constraints. While this is fine for a single type, using arrays of such types waste a lot of memory. To avoid this situation we use a data-oriented technique named Struct of Arrays (SoA). Configuration and state of widgets are separated in several arrays.

Scripting environment

The Berry scripts run on a virtual machine that can communicate with native code via a virtual stack. The virtual stack only accepts specific data types. Therefore, any data transferred from the native code to the virtual machine, or vice versa, must be converted.

A user script is interpreted a single time. Upon widget events, the system invokes function the callbacks declared in the user script. The script interacts with the widget by calling methods that invoke a corresponding function written in native code - also known as native functions. Native function are registered by the system before interpreting any user scripts.

Native functions must be able to query and modify the state of the widgets. To do this, the native function must know which widget is being queried or modified and have access to the widgets’ state pointer. The native function is executed as a method on an object. An object associated to a given widget contains the index of this widget that allows identifying which widget is being queried or modified. The pointer to the widgets’ state is passed through a global variable named _SMOLPHONE_CONTEXT_. We assume that this variable is never used or modified by the user script. Modifying the value of this variable can lead to critical failures, such as segmentation faults. See the following section for more details and possible alternative to avoid this issue.

Pointer passing issues

To query and modify widgets’ state, the native code pass a pointer to the user scripts that in turn pass it to native functions. From a native code perspective, we pass a pointer to a black box, and we retrieve a pointer from this black box. The native code and this black box communicate using Foreign Function Invocations.

Passing and retrieving pointers around FFI boundaries is a pretty bad idea from a Rust perspective. Indeed, Rust has some assumptions about pointers that allow Rust to do many optimizations or simply to ensure memory safety. For example, safe Rust ensures that a mutable reference cannot be aliased and that a pointer is not used beyond its lifespan. This allows Rust to freely move data to different memory locations when it has a mutable access to a data. When Rust passes a pointer to the Virtual Machine it is only mutably borrowed for the time of the invocation. The Virtual machine has no concept of data borrowing and mutability. The VM can hold the reference infinitely. If Rust moves the data in another memory location or deallocates it, this results in a dangling pointer and a possible segmentation fault on pointer dereferencing. On the Rust side, the pointer can be pinned to avoid any move. To avoid deallocation, we could use a pointer with a static lifetime. However, this creates too restrictive constraints that make it impractical. This is why we gave up by introducing a wrapper type Borrowed that basically makes the promise that the pointer that we pass to the Virtual Machine will not be held.

Moreover, a user script can pass any pointer to native functions. This can lead to segmentation faults or undefined behaviors. We assume that the script always pass the expected pointer.

Thus, passing pointers to the Virtual machine is a pretty bad idea. This creates many challenges that are hard and even impossible to solve. Ideally, the pointer should be directly passed to native functions as a context that cannot be accessed and modified by the user scripts. Berry and similar languages don’t afford this feature.

Error handling

Berry scripts have no standard mean of handling errors. Any reported errors cause the end of the execution of the script.

Recipes

Recipes provide instructions to perform a specific task.

Updating Rust

To update to a newer version of Rust, edit the following fields in every Cargo workspace:

• rust-toolchain.toml’s channel field
• Cargo.toml’s rust-version field

Also, update the RUST_VERSION variable in .gitlab-ci.yml to use the same version in CI.

If you update the Rust edition, then you have to update Cargo.toml’s and rustfmt.toml’s edition field.