Technical Introduction
ElectricSQL is a local-first SQL system that adds active-active replication and reactive queries to SQLite. It provides strong database guarantees and works with existing SQLite drivers and libraries.
Under the hood, it uses AntidoteDB for multi-region replication and Rich-CRDTs for invariant safety.
Why would I want ElectricSQL?
Local-first software is resilient, works offline and feels instant to use. It puts the user in control of their data, whilst still enabling realtime collaboration and reactivity.
Ultimately, users just want their software to work. They don’t want to watch a loading spinner and they don’t want their app to freeze because their train has gone into a tunnel. Companies also want their software to just work. In fact, they spend a huge amount of money investing in engineering to make sure it does.
With local-first, the software defaults to just working. Apps don’t freeze when they go into a tunnel. They work when the network is down. They work when the backend is down. Beyond “just working”, it’s also how the app works. There is a quality of user experience that you get from an instantly responsive app that you can’t achieve with the network on your interaction path.
Can’t I do this with existing tech?
There are many different tools to make local-first software. ElectricSQL aims to simplify local-first development by providing a turnkey sync layer that’s compatible with existing open source databases, frameworks and libraries.
- our sync layer provides realtime, low-latency, geo-distributed, multi-user, conflict-free, active-active replication with reactivity, durability, atomic transactions and causal+ consistency with CRDTs
- our system eliminates rollbacks and failure code associated with optimistic writes and provides fully featured relational SQL with referential integrity and constraints
In comparison with other solutions, we provide:
- real SQL with referential integrity and constraints based on Rich-CRDTs
- finality of writes eliminating the need to code failure paths
- fully open-source and integrated with standard SQLite and Postgres
- multi-platform with first class support for mobile, web and edge apps
- drop-in compatible with existing SQL drivers, ORMs and frameworks
- turnkey, conflict-free, active-active replication-as-a-service
- strongest possible consistency for a local-first system
Ultimately, our aim is to cut out the complexity of state transfer by moving it into the database layer, without letting that complexity leak back out into your application code.
How does ElectricSQL work?
ElectricSQL is designed to be used in mobile, web and edge applications, where there are existing SQLite drivers. We provide an ElectricSQL client library that “electrifies” the existing driver library. Electrification means that we pick up on data changes and run a replication process inside the app.
Data is replicated transactionally, as changesets, serialised using protocol buffers, over a web socket interface into the ElectricSQL replication mesh via an Elixir web service. This service writes the data into AntidoteDB, a geo-distributed database that provides transactional causal+ consistency, persistence and operation-based CRDTs. Antidote handles replication between regions. We then have a custom process that tails the Antidote log, turns transactions back into changesets and puts them on the replication web socket.
Our client library process receives and applies the changes, along with any compensations required to preserve integrity. When data changes, the same process dispatches notifications. In your application, you define reactive queries that listen to these notifications and re-query the local SQLite when relevant. If the query results are different, state is updated and components automatically re-render to show live updates in realtime.
Where does Postgres come in?
We also optionally integrate Postgres by consuming and impersonating Postgres logical replication. Specifically, we deploy a Postgres instance into each region, replicating to and from the Antidote in that region. That way you get active-active between Postgres and SQlite, through Antidote.
Postgres can be used to manage data, for example by a admin system or publishing app. It can also be used to run reporting and OLAP queries against data that would otherwise be distributed out across local devices.
What about migrations?
To make this system work, we take over the process of applying DDL migrations. First we pre-process and validate migrations, then we patch them to add automated triggers. This is done using our electric CLI tool, which outputs SQLite and Postgres specific migrations. The SQLite migrations are bundled into an importable JS module. Our client library applies the migrations locally in the app.
This allows the app/user to interact with the SQLite immediately, without needing to sync down any structural migrations. The CLI tool is also used to push migrations to the cloud. There, our management app applies them to the cloud databases. A data record about the migration is written and this is propagated out as a causal dependency of the replication stream.
Right now, we’re limited to additive migrations. Limitations are validated by the CLI tool. In future we will support destructive migrations using lenses.
What are the trade-offs?
Transactional causal+ consistency is a lot stronger than weak eventual consistency. However, it’s still not the same as strong consistency. There is no such thing as a total order in a causally consistent system.
Rich-CRDT techniques like compensations don’t always result in the semantics you’d expect coming from a strongly consistent world view. Adopting local-first requires a shift in mindset to accept the need for merge semantics to resolve overlapping concurrent writes. (A shift that, for what it’s worth, we think fits the world much better than striving for total order in a relativistic universe).
Hopefully this has given you a good mental model of ElectricSQL and an orientation to what’s going on under the hood. If you’d like to have a play with the system, move onto the Quickstart next »