Survival guide: how to migrate from the Firebase Realtime Database to Cloud Firestore

The Firebase Realtime Database has been a popular choice for storing and syncing data in realtime apps. But let‘s face it – it has some significant limitations. Querying is limited, scaling can be tricky, and the data model can easily turn into a tangled mess.

Enter Cloud Firestore, Google‘s next-generation database for mobile and web apps. While it shares some similarities with the Realtime Database – realtime syncing, offline support, simple SDKs – it offers some major upgrades:

• Powerful querying with indexes and compound queries
• Automatic scaling to handle millions of concurrent users
• A more intuitive, collection-oriented data model
• Better offline support across iOS, Android, and web clients

So if you‘re feeling the pains of the Realtime Database, migrating to Firestore is probably a smart long-term bet for your app. But how do you actually do it?

Having helped several clients make the switch, I‘ve put together this survival guide to help you migrate your app from the Realtime Database to Firestore, step-by-step.

Step 1: Re-envision your data model

The first step in migrating to Firestore is rethinking your data structure. While the Realtime Database is just a big JSON tree, Firestore organizes data into documents and collections.

Documents are like JSON objects – they can contain key-value pairs, nested objects, and arrays. Collections are like folders that hold multiple documents.

This model maps more naturally to the kind of data most apps need to store. For example, instead of one big /users node, you‘d have a users collection containing a document for each user.

Some other Firestore data modeling best practices:

Avoid deep nesting. In Firestore, you can nest data in documents, but it‘s generally better to split data into subcollections once you‘re 3-4 levels deep. Otherwise you can hit document size limits.

Flatten your data. Rather than nesting data you often need together, it‘s better to denormalize it (duplicate it) to allow efficient fetching. For instance, store a post‘s content directly in the document rather than in a nested object.

Define your indexes. To enable complex querying, you need to tell Firestore which fields to index (more on this later).

I recommend sketching out the collections and fields you need based on your app‘s screens and access patterns. You may need to restructure things quite a bit from your Realtime Database model.

To make the migration smoother, you can actually keep your Realtime Database running in parallel with Firestore at first. That way, if you miss any data or need to roll back, you still have the old database as a backup.

Step 2: Set up Firestore

Once you‘ve planned out your data model, it‘s time to set up a new Firestore project in the Firebase console. If you want to keep your existing Firebase project, you can simply add Firestore to it.

If you need to migrate existing data from your Realtime Database, Google provides an import/export tool to help with this. You can export your Realtime Database data to a JSON file, then import that directly into Firestore.

Keep in mind, you may need to massage the JSON a bit to fit Firestore‘s data model – like breaking nested data into separate collections. The import tool expects a certain format.

Step 3: Update your app code

Here‘s where the real work happens. You‘ll need to modify your app code in a few key places to read and write from Firestore instead of the Realtime Database.

Change database references. First off, you‘ll need to update any database reference code to point to Firestore instead. So instead of:

// Realtime Database
DatabaseReference ref = FirebaseDatabase.getInstance().getReference("users/123");

You‘ll have:

// Firestore 
DocumentReference ref = FirebaseFirestore.getInstance().collection("users").document("123");

Update data writes. Instead of setValue(), you‘ll use Firestore‘s set() or add() methods to save data. For example:

// Add a new document
Map<String, Object> user = new HashMap<>();
user.put("name", "John Doe");
user.put("email", "[email protected]");

db.collection("users").add(user);

Firestore also offers handy update() methods for changing specific fields without overwriting the whole document.

Modify data reads. This is the biggest change. In the Realtime Database, you attach listeners to get realtime updates whenever data changes.

In Firestore, you can still get realtime updates via addSnapshotListener(). But you‘ll often just want to fetch data once using get():

DocumentReference ref = db.collection("users").document("123");

ref.get().addOnCompleteListener(task -> {
    if (task.isSuccessful()) {
        DocumentSnapshot doc = task.getResult();
        if (doc.exists()) {
            User user = doc.toObject(User.class); 
            // do something with user
        }
    }
});

This code retrieves a single Firestore document and converts it to a custom User object using the toObject() method.

You‘ll need to go through your app and update all data retrieval logic like this to use Firestore APIs. Don‘t forget about error handling!

Rewrite queries. Perhaps the most powerful aspect of Firestore is its querying capabilities. You can easily combine filtering, sorting, and limiting of data.

For example, to find the 10 most recent blog posts by a given user:

Query query = db.collection("posts")
        .whereEqualTo("authorId", "123")
        .orderBy("timestamp", Query.Direction.DESCENDING)
        .limit(10);

This is much simpler and faster than the deep queries and sorting gyrations required in the Realtime Database.

However, Firestore requires explicit indexes for most queries. So you‘ll need to define these indexes in your Firestore console based on the query patterns your app uses. The Firebase documentation has a good guide on this.

Step 4: Test and optimize

Before releasing your migration to the world, be sure to thoroughly test your converted app. You‘ll want to verify that:

• Firestore data is loading correctly throughout the app
• Realtime listeners are triggering as expected when data changes
• Complex queries are returning the right results
• Offline support still works if your app uses it

You should also keep an eye on your Firestore usage in the Firebase console. In particular, watch out for any unexpected spikes in reads or writes, which could indicate an inefficient data model or buggy queries.

Firestore provides some solid profiling tools to help diagnose performance issues. You can see a realtime log of your Firestore activity and pinpoint any slow queries or documents that are too large.

If you find that certain parts of your app are making excessive Firestore reads and writes, consider refactoring your data model or implementing caching layers.

Step 5: Launch!

Once you‘re confident your Firestore migration is rock solid, it‘s go time!

Deploy your updated app to the app stores. In your release notes, it‘s probably wise to mention you‘re migrating databases, just in case any users notice data changes.

Also remember to update any database URL references in your app/website config to point to Firestore. And if you want to be extra careful, you can keep your Realtime Database running read-only as a fallback until you‘re 100% that all data and queries have been ported over.

Last but not least, update your Firebase security rules to the new Firestore rules syntax. The Firebase docs have a handy conversion guide to help with this.

Migrating from the Realtime Database – lessons learned

Having guided multiple clients through Realtime Database to Firestore migrations, here are a few key lessons I‘ve learned:

Take it slow. Migrating databases is a big undertaking. Don‘t feel like you have to move everything over to Firestore at once. A phased, screen-by-screen approach tends to work best.

Beware of fan-out writes. While Firestore makes it easy to update data across multiple documents, be cautious with multi-path writes. They can easily lead to race conditions and make your security rules more complex. Batch writes are safer.

Watch your query costs. Firestore‘s pricing model is based on the number of documents read, so complex queries can rack up your bill if you‘re not careful. Make sure to use limits and define indexes to optimize query performance.

Expect some data modeling trade-offs. No database can perfectly match your app‘s ideal data structure. With Firestore, you‘ll likely need to make some compromises between query efficiency, consistency, and simplicity. Prioritize based on your app‘s specific needs.

Test with production-scale data. It‘s crucial to verify that your new Firestore setup can handle your app‘s actual production load. Generate a realistic dataset to thoroughly stress test queries, security rules, and offline sync before going live.

While migrating from the Realtime Database to Cloud Firestore is a significant effort, it‘s well worth it for the scalability, querying, and developer productivity gains.

By following the steps in this survival guide and learning from those who have gone before you, you too can move your app to Firestore with confidence. Trust me, your future self will thank you!