How to Create a Searchable Developer Log with Gatsby
As developers, we are constantly learning new things – whether it‘s mastering a new programming language, figuring out a tricky bug, or discovering a handy new tool. Taking notes can be a great way to reinforce this knowledge and create a personal knowledgebase to refer back to later.
The challenge is, as our collection of notes and snippets grows over time, it can become difficult to find that one tidbit we vaguely remember writing down months ago. Scattered across various notebooks, docs, and gists, these valuable lessons and solutions get lost like needles in a haystack.
This is where a tool like Gatsby can come to the rescue! Gatsby is a powerful static site generator that makes it easy to create a fast, organized, and easily searchable developer log. By storing log entries as markdown files and leveraging Gatsby‘s rich plugin ecosystem, you can build yourself a personalized, google-like search experience for your coding notes.
Let‘s walk through the process of creating a searchable Gatsby developer log from scratch!
Step 1: Set up a new Gatsby site
First things first, make sure you have the Gatsby CLI installed globally:
npm install -g gatsby-cliNext, we‘ll generate a new Gatsby site. You can start with the default starter template, but I recommend using a pre-designed template like the Novela starter for a slick, polished look right out of the box.
gatsby new my-dev-log https://github.com/narative/gatsby-starter-novelaOnce the site is generated, switch to its directory, install dependencies, and start the development server:
cd my-dev-log
npm install
gatsby developNow if you visit http://localhost:8000 you should see your shiny new Gatsby site! Take a moment to customize the site metadata in gatsby-config.js with your name, bio, social links, etc.
Step 2: Add logging functionality
Now let‘s enable our site to generate log pages from markdown files. First create a new directory at the root of your project called content/logs. This is where we‘ll store all our log entry markdown files.
Each markdown file will require some frontmatter at the top to provide metadata about the log entry. Here‘s an example file content/logs/2020-05-14.md:
---
title: ‘Today I Learned: CSS Grid‘
date: 2020-05-14
tags:
- css
- layout
---
I‘ve been hearing a lot about CSS grid lately so I decided to give it a shot today and WOW - game changer! It makes complex layouts so much simpler than wrestling with floats and positioning. Some key things I learned:
- `grid-template-columns` and `grid-template-rows` define the overall structure
- The `fr` unit lets you divvy up remaining space proportionally
- `grid-gap` adds space between grid cells
- Grid cells are placed automatically but you can use `grid-column` and `grid-row` to manually place them
I followed [Wes Bos‘](https://cssgrid.io/) awesome free video course and I highly recommend it for anyone looking to get started with CSS grid! Notice the frontmatter at the top containing a title, date, and tags for the log entry. The rest is just standard markdown.
To enable Gatsby to generate pages from these markdown files, we‘ll need to install a few plugins:
npm install gatsby-source-filesystem gatsby-transformer-remarkgatsby-source-filesystem will load the markdown files from the filesystem into Gatsby‘s data layer, and gatsby-transformer-remark will parse the frontmatter and convert the markdown to HTML.
Now let‘s add them to our gatsby-config.js:
module.exports = {
// ...
plugins: [
// ...
{
resolve: `gatsby-source-filesystem`,
options: {
name: `logs`,
path: `${__dirname}/content/logs`,
},
},
`gatsby-transformer-remark`
]
}Finally, we need to tell Gatsby how to generate pages from the markdown files. Crack open gatsby-node.js and add the following:
exports.createPages = async ({ actions, graphql, reporter }) => {
const { createPage } = actions
const logTemplate = path.resolve(`./src/templates/log.js`)
const result = await graphql(`
{
allMarkdownRemark(
sort: { order: DESC, fields: [frontmatter___date] }
limit: 1000
) {
edges {
node {
frontmatter {
path
}
}
}
}
}
`)
if (result.errors) {
reporter.panicOnBuild(`Error while running GraphQL query.`)
return
}
result.data.allMarkdownRemark.edges.forEach(({ node }) => {
createPage({
path: node.frontmatter.path,
component: logTemplate,
context: {},
})
})
}This queries for all markdown files, sorted by date descending, and generates a page for each one using a log template component. Speaking of which, we need to actually create that component at src/templates/log.js:
import React from "react"
import { graphql } from "gatsby"
import { Layout, SEO } from "../components"
export default function LogTemplate({ data }) {
const { markdownRemark } = data
const { frontmatter, html } = markdownRemark
return (
<Layout>
<SEO title={frontmatter.title} />
<p>{frontmatter.date}</p>
<div dangerouslySetInnerHTML={{ __html: html }} />
</Layout>
)
}
export const pageQuery = graphql`
query($path: String!) {
markdownRemark(frontmatter: { path: { eq: $path } }) {
html
frontmatter {
date(formatString: "MMMM DD, YYYY")
path
title
}
}
}
`With all that in place, we should now be able to view individual log pages! Try starting up the dev server again and visiting a path like /today-i-learned-css-grid.
However, we still need a way to view a list of all logs. Let‘s update src/pages/index.js to generate that list:
import React from "react"
import { graphql, Link } from "gatsby"
import { Layout, SEO } from "../components"
export default function IndexPage({ data }) {
return (
<Layout>
<SEO title="My Dev Log" />
<ul>
{data.allMarkdownRemark.edges.map(edge => (
<li key={edge.node.id}>
<Link to={edge.node.frontmatter.path}>
{edge.node.frontmatter.title}
</Link>
<span> - {edge.node.frontmatter.date}</span>
</li>
))}
</ul>
</Layout>
)
}
export const pageQuery = graphql`
query {
allMarkdownRemark(sort: { order: DESC, fields: [frontmatter___date] }) {
edges {
node {
id
frontmatter {
date(formatString: "MMMM DD, YYYY")
path
title
}
}
}
}
}
`Now if we visit the homepage of our site, we‘ll see a list of links to all our log entry pages sorted by date! Our basic logging functionality is in place. Time to make it searchable!
Step 3: Implement Search
To add search to our Gatsby log, we‘ll use the gatsby-plugin-elasticlunr-search plugin. This uses elasticlunr.js, a lightweight full-text search engine, to generate a search index of our logs that we can then query.
First install the plugin:
npm install @gatsby-contrib/gatsby-plugin-elasticlunr-searchThen add it to gatsby-config.js and configure which fields to index:
module.exports = {
// ...
plugins: [
// ...
{
resolve: `@gatsby-contrib/gatsby-plugin-elasticlunr-search`,
options: {
fields: [`title`, `tags`, `html`],
resolvers: {
MarkdownRemark: {
title: node => node.frontmatter.title,
tags: node => node.frontmatter.tags,
path: node => node.frontmatter.path,
html: node => node.html,
},
},
},
},
]
}Here we‘re telling it to index the title, tags, and html (the log body content) fields. The resolvers option lets us specify how to grab those values from the markdown nodes.
Now we need to add a search bar component that will allow us to query this index. Let‘s create a new src/components/search.js:
import React, { useState } from "react"
import { Link } from "gatsby"
export default function Search({ index }) {
const [query, setQuery] = useState("")
const [results, setResults] = useState([])
const search = (evt) => {
const query = evt.target.value
index = index.map(({ ref, ...rest }) => ({
id: ref,
...rest,
}));
setQuery(query)
setResults(index.search(query))
}
return (
<div>
<input type="text" value={query} onChange={search} />
<ul>
{results.map(result => (
<li key={result.id}>
<Link to={`/${result.path}`}>{result.title}</Link>
{‘: ‘ + result.tags.join(‘, ‘)}
</li>
))}
</ul>
</div>
)
}This component takes the search index as a prop. When the user types into the search input, it queries the index, formats the results a bit, and renders them as a list of links. Simple but effective!
Now let‘s add this component to our layout. In src/components/layout.js:
import React from "react"
import { graphql, StaticQuery } from "gatsby"
import Search from "./search"
// ...
export default function Layout({ children }) {
return (
<StaticQuery
query={graphql`
query SearchIndexQuery {
siteSearchIndex {
index
}
}
`}
render={data => (
<div>
<header>
<h3>My Dev Log</h3>
<Search searchIndex={data.siteSearchIndex.index} />
</header>
<main>{children}</main>
</div>
)}
/>
)
}Here we‘re using Gatsby‘s StaticQuery component to load the search index, and passing it to the Search component. Gatsby automatically rebuilds the index whenever our markdown files change.
And with that, we have a working search feature for our log! Try it out by starting up the dev server and searching for keywords that appear in your log entries.
Step 4: Enhance the Log
At this point we have a pretty solid foundation for our searchable Gatsby developer log. But there are lots of ways we could enhance it further! Here are a few ideas:
- Syntax highlighting: Make code blocks look snazzy by adding a syntax highlighting plugin like
gatsby-remark-prismjs. - Category pages: Use Gatsby‘s createPages API to generate pages that list all posts in a given tag.
- Comments: Let users leave comments on your posts with a service like Disqus or Utterances.
- Reactions: Add emoji reactions to posts with react-reactions.
- RSS Feed: Generate an RSS feed of your posts so users can subscribe.
- Password Protection: Keep your logs private by adding simple password protection. Check out the
gatsby-plugin-htaccess-password-protectplugin. - Dark Mode: Add a slick dark mode theme option with the
gatsby-plugin-dark-modeplugin.
The beauty of Gatsby is that its extensive plugin ecosystem makes it easy to drop in new features like these without a lot of manual coding. Dive into the Gatsby plugin library to see what else you can add to your log!
Step 5: Deploy the Log
Alright, time to share your awesome new developer log with the world! My favorite way to deploy Gatsby sites is with Netlify.
Netlify connects to your Git repository and automatically builds and deploys your site every time you push new commits. It‘s super simple to set up:
- Push your log‘s code to a new repository on Github.
- Log in to Netlify and click "New site from Git".
- Select your log repo and branch.
- Specify the build command
gatsby buildand the publish directorypublic. - Click "Deploy site"!
Netlify will assign your site a random subdomain like quirky-mclean-a8sc6d.netlify.app. You can change this to a custom domain in the site settings.
Now whenever you add a new log entry, just push your changes to Github and your site will automatically rebuild with the new content. Netlify also generates deploy previews for pull requests which is handy for reviewing changes before merging.
Conclusion
And there you have it! We‘ve walked through the full process of building a searchable, deployable developer log with Gatsby. To recap, we:
- Set up a new Gatsby site using a pre-designed starter template
- Added the ability to generate pages from markdown log entry files
- Implemented search using the elasticlunr plugin
- Explored additional ways to enhance the log experience
- Deployed the log site to the web using Netlify
I encourage you to take the basic foundation we‘ve built here and run with it to create your own personalized developer knowledgebase. The "Today I Learned" approach of logging small learnings each day is a great way to reinforce your knowledge and document your growth as a developer.
Feel free to use my dev-log-starter repo as a jumping off point, and let me know what you build with it! Happy logging!