How to Stand on the Shoulders of Giants: A Developer‘s Guide

"If I have seen further than others, it is by standing on the shoulders of giants." – Isaac Newton

As a developer, it‘s easy to get caught up in the day-to-day grind of coding and forget about the bigger picture. We‘re so focused on solving the immediate problem in front of us that we often neglect one of the most powerful tools at our disposal: the vast wealth of existing knowledge and resources that we can build upon.

In other words, we forget to stand on the shoulders of giants.

The great scientist Isaac Newton understood this concept well. When he wrote in a letter that his work relied on "standing on the shoulders of giants," he was acknowledging the debt he owed to earlier thinkers like Copernicus, Galileo, and Kepler. Newton‘s groundbreaking work on gravity, optics, and calculus wasn‘t conjured out of thin air – it was built on the foundation laid by these "giants."

This is a lesson that every developer, whether working on cutting-edge artificial intelligence or a humble web app, should take to heart. No matter what problem you‘re trying to solve, chances are that someone, somewhere, has already made some progress on it. By seeking out and leveraging this existing work, you can see further, work smarter, and ultimately achieve much more than you could alone.

Great Developers Stand on Giant Shoulders

The history of computing is a story of building on previous breakthroughs. Even the most revolutionary technologies have their roots in earlier ideas and inventions.

Take the internet, for example. While we think of it as a modern marvel, its core technologies, like packet switching and TCP/IP, were developed in the 1960s and 70s by pioneers like Donald Davies, Paul Baran, and Vint Cerf. The World Wide Web, invented by Tim Berners-Lee in 1989, was built on top of this foundation.

Or consider the field of artificial intelligence. Many of the key ideas, like neural networks and machine learning, have been around for decades. The perceptron, a simple type of neural network, was invented by Frank Rosenblatt in 1958. Backpropagation, the workhorse algorithm of modern AI, was developed in the 1970s by researchers like Paul Werbos and Geoffrey Hinton.

Today‘s AI breakthroughs, from DeepMind‘s AlphaGo to OpenAI‘s GPT-3, are the result of building on this long history of research. As Hinton himself said, "The neural networks we use today are very similar to the ones we were developing in the 1980s. What‘s changed is that we now have more data and more computing power, so we can train much bigger models."

This pattern of building on existing work is especially clear in the world of open source software. According to a 2020 report from Synopsys, 99% of commercial codebases contain open source components. On average, open source makes up 70% of a codebase.

Take a look at any popular software project on GitHub, and you‘ll find a "package.json" or "requirements.txt" file listing dozens or even hundreds of dependencies. Modern web development frameworks like React and Angular are built on a complex ecosystem of libraries for everything from state management to animation. Data science tools like NumPy, Pandas, and scikit-learn are the building blocks of countless AI and analytics applications.

By leveraging these existing tools and libraries, developers can be orders of magnitude more productive than coding everything from scratch. As Tom Preston-Werner, co-founder of GitHub, put it: "Open source is the gravitational center of coding. It‘s happening at a faster pace than ever before…If you‘re not open sourcing, you‘re falling behind."

Avoiding Not-Invented-Here Syndrome

Despite the clear benefits of building on existing work, many developers and organizations still fall prey to "Not Invented Here" (NIH) syndrome. This is the misguided belief that in-house, custom-built solutions are inherently superior to using external code or tools.

NIH syndrome can stem from a variety of factors, such as:

• Pride in one‘s own abilities and a desire to prove oneself
• A misunderstanding of the full cost (time, bugs, maintenance) of building something from scratch
• Concerns about the licensing, security, or reliability of external code
• A desire for total control and customization of every part of the stack

In some cases, building in-house can be the right call. If you‘re working on something truly novel, or if you have very specific requirements not met by existing solutions, it may be necessary to roll your own.

But more often than not, NIH leads to wasted time and subpar results. Developers spend months reinventing wheels while falling behind on their actual goals. Custom-built solutions often end up buggier, less performant, and harder to maintain than mature open source alternatives.

As a cautionary tale, consider the case of Knight Capital Group. In 2012, this financial services firm lost over $460 million in just 45 minutes due to a bug in their custom-built high frequency trading software. The glitch caused their system to execute a huge number of erroneous trades, leading to massive losses. While the exact cause was never made public, many experts speculate that Knight‘s reliance on in-house software, with minimal use of standard libraries and inadequate testing, was a major contributing factor.

Finding the Right Shoulders to Stand On

So how can developers avoid NIH syndrome and make the most of existing resources? The first step is knowing where to look. Here are some of the best places to find "giant shoulders" to stand on:

• GitHub: The go-to platform for open source software, with over 100 million repositories. Search for projects related to your domain, and look at the "Used by" and "Dependencies" graphs to find popular, well-maintained libraries.

• Package Managers: Tools like npm, PyPI, and Maven make it easy to discover and use libraries in your language of choice. Sorting by download count or star rating can help identify the most widely used packages.

• Research Papers: For more academic topics, sites like Google Scholar, arXiv, and Semantic Scholar are invaluable resources. Look for highly cited papers in your field, and follow their citations to find related work.

• Conference Talks: Events like PyCon, Strange Loop, and OpenAI‘s Scholars Conference are great places to learn about cutting-edge work from leading experts. Many conferences post video recordings of talks online.

• Blogs and Newsletters: Following blogs and newsletters from thought leaders in your industry can help you stay on top of new ideas and tools. Some of my favorites include Adrian Colyer‘s "The Morning Paper", Denny Britz‘s "WildML", and Sergey Zhuk‘s "Data Machina".

Once you‘ve identified some promising resources, it‘s important to evaluate them carefully before building on top of them. Some key factors to consider:

• License: Is the code available under a permissive open source license (e.g. MIT, Apache)? If not, are you able and willing to comply with the terms of the license?

• Maintenance: Is the project under active development, with recent commits and responses to issues? Unmaintained code can become a liability over time.

• Popularity: While not always a perfect metric, a project‘s popularity (stars, forks, downloads) can be a good indicator of its quality and reliability. More users means more bug reports and more battle testing.

• Documentation: Is the project well-documented, with clear examples and API references? Good documentation makes it much easier to get up and running.

• Test Coverage: Does the project have a comprehensive test suite? Automated tests are essential for catching bugs and ensuring the code works as advertised.

Once you‘ve found a solid foundation to build on, the real work begins. It‘s not enough to just drop in a library and call it a day – you need to really understand how it works under the hood. Read the source code, step through it in a debugger, and experiment with different configurations and edge cases.

This deep understanding is what allows you to use the tool effectively, adapt it to your specific needs, and troubleshoot issues when they arise. It‘s also what enables you to contribute back to the project, whether through bug reports, feature requests, or even pull requests.

Case Study: Building a Recommender System

To illustrate these principles, let‘s walk through an example of building a machine learning application on top of existing tools and research. Specifically, we‘ll look at creating a recommender system for an e-commerce site.

Recommender systems are a common application of machine learning, used by companies like Amazon, Netflix, and Spotify to suggest items to users based on their past behavior. Building a recommender from scratch would be a daunting task, requiring expertise in areas like data processing, algorithm design, and model evaluation.

Fortunately, we don‘t have to start from zero. There‘s a wealth of existing resources we can leverage:

• Research Papers: The field of recommender systems has been actively studied for over two decades. Seminal papers like "Item-Based Collaborative Filtering Recommendation Algorithms" by Sarwar et al. and "Matrix Factorization Techniques for Recommender Systems" by Koren et al. provide a solid theoretical foundation.

• Open Source Libraries: Tools like Python‘s Surprise and implicit libraries provide well-tested implementations of popular recommendation algorithms like collaborative filtering and matrix factorization. Apache Spark‘s MLlib includes a suite of scalable recommendation algorithms.

• Public Datasets: Datasets like MovieLens and LastFM are widely used in recommender system research and make great starting points for prototyping and benchmarking new models.

• Cloud Platforms: Managed services like Google Cloud‘s Recommendation AI and AWS Personalize make it possible to build and deploy recommender systems without managing infrastructure.

By standing on these shoulders, we can quickly get a working recommender system up and running, and then iterate and customize it for our specific use case. For example, we might start by using Surprise to train a baseline collaborative filtering model on the MovieLens dataset. From there, we could experiment with different algorithms, incorporate additional data sources (e.g. item metadata, user demographics), and tune hyperparameters to improve performance.

Once we have a model we‘re happy with, we can use a tool like MLflow to package it for production and deploy it on a platform like Google Cloud AI Platform. And we can continue to monitor and update the model over time as new data comes in.

Throughout this process, we‘re not just blindly using off-the-shelf components, but really understanding how they work and adapting them to our needs. And we‘re not afraid to dive into the source code, read the original research papers, and experiment with new approaches.

Conclusion

In the fast-moving world of software development, it‘s easy to get caught up in the hype of the latest and greatest tools and frameworks. But the most successful developers and teams are often those who understand and leverage the work that has come before.

By standing on the shoulders of giants – the researchers, engineers, and open source contributors who have paved the way – we can see further, work smarter, and ultimately achieve more than we ever could alone. This doesn‘t mean blindly using off-the-shelf solutions, but rather deeply understanding and adapting existing work to our specific needs.

Of course, there will always be times when we need to innovate and build something completely new. But even then, we should be looking for opportunities to learn from and build on prior art. As Isaac Newton himself said, "We build too many walls and not enough bridges."

So the next time you‘re faced with a challenging problem, remember to look for the giants. Chances are, they‘re waiting to give you a boost.