SMPI is a React Native library that allows you to use SQLite from your Javascript code with identical behavior across platforms.

What is SMPI?

SMPI stands for SQLite Message Passing Interface.

It allows you to use SQLite in cases when the client is not the same process that embeds the SQLite FFI.

For example:

• Two processes on the same OS.
• Two processes over a network.

SMPI consists of 3 components that work together:

• 1. Binary.
• 2. IO Provider (IOP).
• 3. Client.

1. Binary

The Binary essentially wraps the standard SQLite FFI functions, and exports a FFI with the functions:

• input(inputMsg)
  • Takes a JSON string as input.
• setOutputCb(cbFn)
  • Takes a C pointer to a callback function, that is called with a JSON string as an argument representing output.

This converts the standard SQLite C FFI into a an IO stream - where events can flow in both directions.

JSON is used to represent messages, and allows data to be passed between processes.

2. IO Provider

IO Providers pass messages between the Binary and Client.

Different platforms, run times and languages will require different implementations of IOPs.

Each environment has the general ability to:

• Use a C FFI.
• Send and receive data/messages.

IOPs implement specific code for these general abilities, and encapsulates them so that the Client only has to expect a const outputMsg = await send(inputMsg) interface.

The IOP generally does not parse the JSON messages, it only passes strings between the Binary and the Client.

Current IOPs include React Native and Node.js.

3. Client

The Client is the code API used in the application that uses SQLite.

Only Javascript is supported as a Client currently.

So this is a server database like MySQL?

Yes in the sense that is passes messages, there is a client and a server.

But No in the sense that SMPI is designed to be used in the same context as SQLite (but in a single local application that uses many OS processes/threads).

E.g. Instead of sending all requests for all users of an application to a single central remote database, each user has their own local database.

Also the SMPI runtime is a single background thread, not a standalone process. The Host process that embeds the SMPI FFI is the parent of this background thread, they communicate over the FFI.

There is no TCP/IP or Unix socket communication in the SMPI Binary, but you could use networking libraries in your Host process to route SMPI messages over the network for some custom use case.

What problem does SMPI solve?

Issues with current React Native SQLite libraries:

• A. Different SQLite versions and extensions.

  • SQL language features and extensions may be missing from some platforms, resulting in your app not working.

• B. Two or more implementations of one program.

  • Android and iOS both implement the same state machine with two different code bases.

    • E.g:

      • Java/Kotlin for Android.
      • Objective C/Swift for iOS.

    • This means effort must be spent to ensure both programs behave the same, and is a possible source of hard to understand bugs.

• C. No transaction support.

  • Because SQLite has many runtime modes which change the meaning of locks and transactions, it can get complicated to understand the behavior of your app.

    • This is especially true when the SQLite operations are being made from JS with async function calls, and differing native code for each platform may order/process the operations differently.

  • Expo has transaction in their SQLite API, but never actually starts a transaction leaving your application state possibly incorrect as read/writes are not isolated and/or atomic.

How SMPI solves these issues.

• A. The same SQLite version and extensions on all platforms.

  • SMPI bundles SQLite into the binary and does not try to use the system SQLite.

• B. One program cross compiled.

  • The SMPI Binary is written in Rust and cross compiled for each platform, which means behavior and state transitions are identical on every platform.

• C. First class transaction support.

  • SMPI enforces a single runtime mode (WAL) so the meaning of locks and transactions never changes.

  • Transactions are tested at 3 points:

    • Binary: Directly in Rust code.
    • IOP: At the FFI.
    • Client: JS based tests.

  • The SMPI JS Client has a clear transaction based API.

Summary

SMPI:

• Enables using SQL for application state.
• Provides a clear high level API that is quick to install, understand and use.
• Allows writing a single application that runs on many platforms with identical results.
• Removes hard to debug cross platform issues.