Building a GitHub Repo Explorer with React and Elasticsearch

Introduction

As developers, we often find ourselves searching for code examples, open source libraries, and side project inspiration on GitHub. With over 200 million repositories, GitHub is an invaluable resource – but exploring it can sometimes feel like searching for a needle in a haystack.

In this tutorial, we‘ll harness the power of Elasticsearch and ReactiveSearch to build our own GitHub repository explorer app in React. Elasticsearch will allow us to quickly search through a large volume of repository data, while ReactiveSearch will generate the search UI components for us and simplify wiring everything together.

By the end, we‘ll have a fully functional web app where users can search for repos by keyword and filter the results based on factors like programming language, topic tags, number of stars, number of forks, and more. Let‘s get started!

What is Elasticsearch?

Elasticsearch is a distributed, RESTful search engine built on top of Apache Lucene. It allows you to store, search, and analyze huge volumes of structured and unstructured data in near real-time.

Some key features of Elasticsearch include:

• Full-text search
• Structured search
• Analytics
• Distributed architecture supporting high availability and scalability
• RESTful API accessed using JSON over HTTP

Under the hood, Elasticsearch uses inverted indices to quickly look up which documents contain the search terms. An inverted index is like the index at the back of a book – it maps each unique search term to all the documents and positions within documents where that term appears. This allows Elasticsearch to efficiently find matching documents without having to scan through all the text.

Elasticsearch is used by many well-known companies including Wikipedia, StackOverflow, GitHub, and Netflix to power search and analytics in their applications. It has become the de facto standard for search engines due to its speed, scalability, and powerful query DSL.

Simplifying search UI development with ReactiveSearch

While Elasticsearch provides a powerful API for searching data, integrating it into a UI typically involves a lot of boilerplate code to manage the query state, pagination, URL params, etc. This is where ReactiveSearch comes to the rescue.

ReactiveSearch is a React UI components library that connects to Elasticsearch to provide pre-built, customizable search UI components. It intelligently manages the search state and provides scaffolding for common search, filtering, and results display patterns.

Here are some of the benefits of using ReactiveSearch:

• Works out of the box with any Elasticsearch index
• 30+ pre-built components for search, filtering, results, etc.
• Bring your own UI – customize the look and feel of components
• Simplified data flow – components manage the search state for you
• Built in support for URL routing, pagination, infinite scroll, etc.

ReactiveSearch currently provides components for React, React Native, Vue.js and vanilla JavaScript. In this tutorial, we‘ll be using the React web components.

Building the GitHub Repo Explorer

Now that we have an understanding of the tools we‘ll be using, let‘s dive into building the app! We‘ll break it down into the following steps:

1. Setting up the project
2. Importing GitHub repo data into Elasticsearch
3. Building the search UI
4. Styling the UI components
5. Deploying the app

1. Setting up the project

We‘ll use Create React App to quickly bootstrap a new React project. Make sure you have Node.js version 14.0.0 or later and npm version 5.6 or later installed. Open up your terminal and run:

npx create-react-app github-repo-explorer
cd github-repo-explorer

Next, let‘s install the ReactiveSearch dependency:

npm install @appbaseio/reactivesearch

We‘ll also be using the Font Awesome icon pack for some of our UI elements, so let‘s add the CDN link to public/index.html:

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.4/css/all.min.css" integrity="sha512-1ycn6IcaQQ40/MKBW2W4Rhis/DbILU74C1vSrLJxCq57o941Ym01SwNsOMqvEBFlcgUa6xLiPY/NS5R+E6ztJQ==" crossorigin="anonymous" referrerpolicy="no-referrer" />

2. Importing GitHub repo data

For the purposes of this tutorial, I‘ve prepared a sample dataset of about 5000 popular GitHub repositories that we can import into Elasticsearch and use to power our app.

The dataset is hosted for free on Elastic Cloud, which provides managed Elasticsearch hosting. You can sign up for a free 14-day trial here. Once you have an account, create a new deployment and take note of your Cloud ID and password.

Next, navigate to the "Indices" section in the left sidebar and click "Create index". Give your index a name like github-repos and click "Create" to initialize it.

Now we need to add our sample data to the index. Click the "Add data" button at the top of the page and select "Upload a file". Download the sample JSON dataset from here, choose it as your file, and click "Import" to index the documents.

After a few seconds, you should see a message that the data was imported successfully. We now have a populated Elasticsearch index ready to integrate into our app!

3. Building the search UI

Here is the basic folder structure we will use for our app:

src
├── App.css
├── App.js  
├── index.css
└── index.js

App.js will contain the bulk of our application logic. Let‘s open it up and start building out the UI with ReactiveSearch components:

import React from ‘react‘;
import ‘./App.css‘;
import { ReactiveBase, DataSearch, ReactiveList } from ‘@appbaseio/reactivesearch‘;

function App() {
  return (
    <ReactiveBase
      app="github-repos"
      url="https://user:[email protected]"
    >
      <h2>GitHub Repository Explorer</h2>

      <DataSearch
        componentId="search"
        dataField={[
          ‘name‘,
          ‘description‘,
          ‘owner‘,
          ‘topics‘
        ]}
        placeholder="Search repositories"
      />

      <ReactiveList
        componentId="results"
        dataField="name"
        react={{
          and: [‘search‘, ‘filters‘]
        }}
        pagination={true}
        size={6}
      />

    </ReactiveBase>
  );
}

export default App;

Here‘s what‘s going on:

• We import the ReactiveSearch components we need from the library
• The ReactiveBase component is the root component that connects the child components to our Elasticsearch index
  • app prop specifies the name of our index
  • url prop should be your Elastic Cloud endpoint URL
• Inside ReactiveBase, we add a heading and two other components:
  • DataSearch renders a search input and provides autocomplete suggestions
    • componentId uniquely identifies this component
    • dataField specifies which index fields to search across
    • placeholder is the greyed out text shown when the input is empty
  • ReactiveList displays the search results in a list format
    • react prop tells it to respond to changes in the search and filters components
    • pagination adds a pagination navigation to the result list

The DataSearch component should now allow you to type in a search query and see matching results in the ReactiveList!

Let‘s add some more search controls for the user. Update your App.js file:

// ...

// Add more imports 
import {
  MultiList, 
  SingleRange
} from ‘@appbaseio/reactivesearch‘;

function App() {
  return (
    // ...
    <div>
      <MultiList
        componentId="language-list"  
        dataField="language.keyword"
        title="Filter by language"
        size={100}
        sortBy="count"
        queryFormat="or"
        showCheckbox={true}
        showCount={true}
        react={{
          and: [‘search‘, ‘filters‘]
        }}
        filterLabel="Languages"
      />

      <MultiList
        componentId="topics-list"
        dataField="topics.keyword"
        title="Filter by topics"
        size={50}
        sortBy="count"
        queryFormat="or"
        showCheckbox={true}
        showCount={true}  
        react={{
          and: [‘search‘, ‘filters‘, ‘language-list‘]
        }}
        filterLabel="Topics"
      />

      <SingleRange
        componentId="stars-filter"
        dataField="stars"
        title="Repo stars"
        data={[
          { "start": 0, "end": 100, "label": "< 100" },
          { "start": 100, "end": 1000, "label": "100 - 1K" },
          { "start": 1000, "end": 10000, "label": "1K - 10K" },
          { "start": 10000, "end": null, "label": "> 10K" },
        ]}
        showRadio={true}
        react={{
          and: [‘search‘, ‘filters‘, ‘language-list‘, ‘topics-list‘]
        }}
        filterLabel="Stars" 
      />

      { /* ... */ }

      <ReactiveList
        react={{
          and: [‘search‘, ‘filters‘, ‘language-list‘, ‘topics-list‘, ‘stars-filter‘]  
        }}
      />
    </div>
  )
}

Here we‘ve added some additional filters:

• Two MultiList components to filter by programming language and topic tags
  • These are basically selectable lists that filter down the results
  • dataField refers to the keyword-mapped fields for these properties
  • sortBy controls how the list items are sorted
• A SingleRange component to filter repos by star count ranges
  • data defines the specific ranges the user can select from

We‘ve also updated the ReactiveList to respond to changes in all of these new filter components by adding their componentIds to the react prop array.

At this point, we have a functional faceted search interface for exploring GitHub repos! Users can enter searches, filter by language/topic/stars, and combine multiple filters together.

4. Styling the UI components

Optionally, you may wish to apply some custom CSS to the UI components to match your desired look and feel. ReactiveSearch allows you to supply CSS class names for the different parts of each component using the innerClass prop.

For example, to style the DataSearch component:

<DataSearch
  innerClass={{
    title: ‘search-title‘,
    input: ‘search-input‘
  }}
/>

And in your CSS file:

.search-title {
  font-size: 24px;
  margin-bottom: 10px;
}

.search-input {
  height: 40px;
  padding: 5px 10px;
  min-width: 300px;
}

Feel free to experiment with the UI and customize it to your liking! You can refer to the ReactiveSearch docs for all the different class names you can target.

5. Deploying the app

When you‘re ready to share your app with the world, you can easily deploy a production build to a static hosting service like GitHub Pages or Netlify.

Create a production build of your app using:

npm run build

This compiles your app into a build folder that is ready to be deployed to any static file host.

GitHub Pages makes it dead simple to host websites for free from a GitHub repository. Assuming you‘ve been committing your changes to a repo:

1. Create a new branch called gh-pages
2. Remove the build directory from .gitignore and commit it
3. Push your gh-pages branch to GitHub

That‘s it! Your app is now accessible at https://<username>.github.io/<repo-name>.

Conclusion

Congratulations, you now have a working GitHub repo search app powered by React, Elasticsearch, and ReactiveSearch!

In this tutorial, we covered:

• How Elasticsearch enables fast, scalable full-text search
• Using ReactiveSearch to quickly build a faceted search UI in React
• Indexing a dataset of GitHub repos into Elasticsearch
• Creating search and filter controls for language, topic, stars, etc.
• Styling the UI components
• Deploying a production build of the app

You now have the skills to build your own search experiences with React and Elasticsearch. Here are some ideas for additional features you could add:

• Sorting controls (e.g. by stars, last updated, etc.)
• "Details" page for each search result with more metadata
• Aggregations / analytics (e.g. repos per language)
• User authentication / private indexes
• Support for code search within the repos
• Highlighting of search terms in the results

There are endless possibilities to enhance and extend the app. If you‘d like to see a fully built out version of the Repo Explorer, check out this CodeSandbox.

I hope this tutorial has been helpful in your React and Elasticsearch learning journey. Feel free to leave a comment if you have any questions or feedback. Happy coding!