Supercharging Developer Productivity With Advanced Code Search

Rate this content
Bookmark

Google & Facebook Engineers are able to search over their gargantuan codebase using an internal code search engine. The search engine accelerates the ability of their developers (both old & new) to understand any part of their codebase & start contributing immediately! What about the rest of us?

In this talk, I'll walk through the different types of code search, tools & software, and advanced tips & tricks to navigate any type of codebase easily. With the advent of large code repositories and sophisticated search capabilities, code search is increasingly becoming a key software development activity. Every developer in the world that spends an insane amount of time reading rather than writing code should have access to the best code search tools that amplifies their productivity. The audience will walk away with everything they need to confidently onboard, navigate & understand any small, medium-sized and daunting codebase.

This talk has been presented at React Advanced Conference 2021, check out the latest edition of this React Conference.

FAQ

Sourcegraph is a search engine designed for code, allowing developers to search across over 2.1 million open-source repositories on GitHub and GitLab as well as private repositories. It is often referred to as the Google of code search.

Sourcegraph supports three types of searches: literal search, regular expression search, and structural search, each catering to different search needs and complexities in codebase navigation.

Key features of Sourcegraph include precise code intelligence, batch changes for automating pull requests, code monitoring, and code insights. These tools are designed to improve developer productivity and code management.

Sourcegraph facilitates faster onboarding and feature development by allowing developers to quickly navigate and understand extensive codebases, making code more accessible and easier to manage across multiple projects.

The batch changes feature in Sourcegraph allows developers to create a file that can make multiple pull requests across different repositories automatically, helping to streamline code updates and project management.

Yes, Sourcegraph can perform targeted searches within specific repositories or branches. Users can specify the repository or branch as part of their search query to narrow down results.

Sourcegraph aids in refactoring by identifying references to specific components or code across all connected repositories. This helps ensure consistency and accuracy during the refactoring process.

You can learn more about using Sourcegraph and its features at learn.sourcegraph.com, which offers a suite of learning tutorials covering various aspects of code search and management.

Prosper Otemuyiwa
Prosper Otemuyiwa
10 min
22 Oct, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Developers can use Sourcegraph to easily search for code using literal, regular expression, and structural patterns. Sourcegraph allows searching through open source and private code indexed across GitHub and GitLab, and supports searching for unstable batch API in React.js. It also enables searching with regular expressions, finding dependencies and usage patterns. With structural search, developers can search for specific blocks of code and narrow down their search using keywords. Sourcegraph is essential for developer velocity, helping teams quickly search through repositories, find references to shared systems, and providing powerful search parameters.

1. Introduction to Sourcegraph and Code Search

Short description:

Developers spend a lot of time reading and writing code, so they need tools to easily search for code. Sourcegraph is a search engine for open source and private code. It offers features like local search, code intelligence, batch changes, code monitoring, and code insights. With Sourcegraph, developers can search using literal, regular expression, and structural patterns.

All right, so if you're on Sourcegraph.com, just follow me. I'm Prosper Otemiyuwa. I work as a Developer Advocate at Sourcegraph. We have an agenda.

So every developer in the world spends a huge amount of time reading and writing code. In fact, if you use GitHub a lot, you discover that even when you're not with your laptop, you're on GitHub, you're trying to get notifications for pull requests, comments, and every other thing, you're doing code, right? So if you spend more time reading and writing, then you should have the tools that allow you to search for code very easily.

So we have a tool called Sourcegraph. Sourcegraph.com. That's why I said if you open your phones. If you open your phones and go to Sourcegraph.com, I like to call Sourcegraph.com the Google of code search. So this is how it looks like. If your interface doesn't look like that, you're not on Sourcegraph, please. So just cross check.

So with Sourcegraph, this is the value we offer to developers. Sourcegraph has currently indexed over 2.1 million open source repositories across GitHub and GitLab. So right now, you can search code on Sourcegraph and it gives you data from GitHub and GitLab, right? And you can also set private code across several repositories. With Sourcegraph, you can do a local search on your local IDs. You can have precise code intelligence on Sourcegraph.com. And there's a feature we have called batch changes. Instead of automating, instead of, you know, opening several pull requests to different with batch changes, you can have a file and then it can make several pull requests for you. Instead of you having to do that yourself. So it's called batch changes. And then we have two features called code monitoring and code insights. But for this talk, I'm going to be talking more about the search.

All right. So, like I said, Sourcegraph can be called Google of code search. It's literally a search engine that allows you to search all of open source code and all of your private code, right? So what are the code search patterns we have with Sourcegraph? There are three types of search you can do right now. Literal, regular expression, and structural. So let's go to literal. I know many of us are familiar with the source space component, right? So if you're onboarding a new developer or you get onboarded to a codebase and you're trying to see occurrences of a particular class, symbol, or definition, you can literally copy the codebase and paste it into Sourcegraph and then it goes ahead to search for you.

2. Sourcegraph Search Capabilities

Short description:

Sourcegraph allows you to search all the open source code indexed across GitHub and GitLab. You can search through divs, time difference, committers, and APIs. It supports searching for the unstable batch API in React.js and provides results from within the React code and organization. Sourcegraph also enables searching with regular expressions, allowing you to find dependencies and usage patterns. You can easily see how many people are using your open-source packages or specific versions. Additionally, you can search through branches and connect private repositories. Structural search is another powerful feature of Sourcegraph.

So right now, this is me placing the source space components in Sourcegraph, and it searches all of the open source code it has indexed across GitHub and GitLab. Examples are results of people that are using source space across this GitHub and GitLab. So now you can see the responses. You can scroll down. On the left-hand side, you can see all the filters and operators that you can add.

So you can search through divs, you can search through the time difference, you can also search through the committers of the code, and you can have access to how people are using certain APIs. Another API that you can talk about in React JS2 is the unstable batch of this. So you want to know how many people are using unstable batch of this in the world? How many people are really using it, and how are they using this particular API method? With source graph, all you need to do is paste it in. So if you look at the arrow, it searches within the React.js organisation and it brings you out results of every usage of unstable batch of this. From within the React code itself and every other example within the React organisation, you can get results.

Now let's go to regular expression. How many of us are familiar with regular expression? I know we probably don't use it every day, but with source graph, it allows you to flex your muscles with regular expressions. So if you understand what you're trying to look for, you're trying to look for something within your file, for example. Here I was trying to look for how people are using some of the latest versions of React by using regular expression to search via the packages of several repositories. So in the source graph search engine, I toggled on the regular expression mode, and these are the results it gives me. So with these results, you can see and you can narrow it down to get the results of how many people or how many projects are dependent on a particular version of React. So some of us here are open source maintainers, others are open source authors. You want to get an idea of how many people are using your open-source package or how many people are using certain versions because you need to deprecate certain methods or because there is a new architecture you need to work on. So with this, you can easily scroll and see how people are using the webpack, Remix Run, Visualizer. You can see how many people are dependent on a particular version of your project.

This is an example of me searching through branches. I was searching through the source graph repository, and specifying with regular expression to search within every branch that has mchap within the name of the branch. So this goes into all of the branches that have mchap within it and then searches for Ubuntu latest. With that it returns all the codebase for me. This is just open-source code. Then connect all your private repositories as well. The same results it gets you. It just searches within the scope of all the repositories that you've connected to source graph. And then we have the structural search. I think this is one of my best types of search.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Modern Web Debugging
JSNation 2023JSNation 2023
29 min
Modern Web Debugging
Top Content
This Talk discusses modern web debugging and the latest updates in Chrome DevTools. It highlights new features that help pinpoint issues quicker, improved file visibility and source mapping, and ignoring and configuring files. The Breakpoints panel in DevTools has been redesigned for easier access and management. The Talk also covers the challenges of debugging with source maps and the efforts to standardize the source map format. Lastly, it provides tips for improving productivity with DevTools and emphasizes the importance of reporting bugs and using source maps for debugging production code.
The Future of Performance Tooling
JSNation 2022JSNation 2022
21 min
The Future of Performance Tooling
Top Content
Today's Talk discusses the future of performance tooling, focusing on user-centric, actionable, and contextual approaches. The introduction highlights Adi Osmani's expertise in performance tools and his passion for DevTools features. The Talk explores the integration of user flows into DevTools and Lighthouse, enabling performance measurement and optimization. It also showcases the import/export feature for user flows and the collaboration potential with Lighthouse. The Talk further delves into the use of flows with other tools like web page test and Cypress, offering cross-browser testing capabilities. The actionable aspect emphasizes the importance of metrics like Interaction to Next Paint and Total Blocking Time, as well as the improvements in Lighthouse and performance debugging tools. Lastly, the Talk emphasizes the iterative nature of performance improvement and the user-centric, actionable, and contextual future of performance tooling.
Remix Flat Routes – An Evolution in Routing
Remix Conf Europe 2022Remix Conf Europe 2022
16 min
Remix Flat Routes – An Evolution in Routing
Top Content
Remix Flat Routes is a new convention that aims to make it easier to see and organize the routes in your app. It allows for the co-location of support files with routes, decreases refactor and redesign friction, and helps apps migrate to Remix. Flat Folders convention supports co-location and allows importing assets as relative imports. To migrate existing apps to Flat Routes, use the Remix Flat Routes package's migration tool.
pnpm – a Fast, Disk Space Efficient Package Manager for JavaScript
DevOps.js Conf 2022DevOps.js Conf 2022
31 min
pnpm – a Fast, Disk Space Efficient Package Manager for JavaScript
pnpm is a fast and efficient package manager that gained popularity in 2021 and is used by big tech companies like Microsoft and TikTok. It has a unique isolated node module structure that prevents package conflicts and ensures each project only has access to its own dependencies. pnpm also offers superior monorepo support with its node module structure. It solves the disk space usage issue by using a content addressable storage, reducing disk space consumption. pnpm is incredibly fast due to its installation process and deterministic node module structure. It also allows file linking using hardlinks instead of symlinks.
How to Make a Web Game All by Yourself
JS GameDev Summit 2023JS GameDev Summit 2023
27 min
How to Make a Web Game All by Yourself
This talk guides you on how to make a web game by yourself, emphasizing the importance of focusing on tasks that interest you and outsourcing the rest. It suggests choosing a game engine that allows distribution on the web and aligns with your understanding and enjoyment. The talk also highlights the significance of finding fun in the creative process, managing scope, cutting features that don't align with the game's direction, and iterating to the finish line. It concludes by discussing the options for publishing the game on the web and leveraging unique web features.
Atomic Deployment for JS Hipsters
DevOps.js Conf 2024DevOps.js Conf 2024
25 min
Atomic Deployment for JS Hipsters
This Talk discusses atomic deployment for JavaScript and TypeScript, focusing on automated deployment processes, Git hooks, and using hard links to copy changes. The speaker demonstrates setting up a bare repository, configuring deployment variables, and using the post-receive hook to push changes to production. They also cover environment setup, branch configuration, and the build process. The Talk concludes with tips on real use cases, webhooks, and wrapping the deployment process.

Workshops on related topic

React, TypeScript, and TDD
React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
Paul Everitt
Paul Everitt
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
Mastering advanced concepts in TypeScript
React Summit US 2023React Summit US 2023
132 min
Mastering advanced concepts in TypeScript
Top Content
Featured WorkshopFree
Jiri Lojda
Jiri Lojda
TypeScript is not just types and interfaces. Join this workshop to master more advanced features of TypeScript that will make your code bullet-proof. We will cover conditional types and infer notation, template strings and how to map over union types and object/array properties. Each topic will be demonstrated on a sample application that was written with basic types or no types at all and we will together improve the code so you get more familiar with each feature and can bring this new knowledge directly into your projects.
You will learn:- - What are conditional types and infer notation- What are template strings- How to map over union types and object/array properties.
Integrating LangChain with JavaScript for Web Developers
React Summit 2024React Summit 2024
92 min
Integrating LangChain with JavaScript for Web Developers
Featured Workshop
Vivek Nayyar
Vivek Nayyar
Dive into the world of AI with our interactive workshop designed specifically for web developers. "Hands-On AI: Integrating LangChain with JavaScript for Web Developers" offers a unique opportunity to bridge the gap between AI and web development. Despite the prominence of Python in AI development, the vast potential of JavaScript remains largely untapped. This workshop aims to change that.Throughout this hands-on session, participants will learn how to leverage LangChain—a tool designed to make large language models more accessible and useful—to build dynamic AI agents directly within JavaScript environments. This approach opens up new possibilities for enhancing web applications with intelligent features, from automated customer support to content generation and beyond.We'll start with the basics of LangChain and AI models, ensuring a solid foundation even for those new to AI. From there, we'll dive into practical exercises that demonstrate how to integrate these technologies into real-world JavaScript projects. Participants will work through examples, facing and overcoming the challenges of making AI work seamlessly on the web.This workshop is more than just a learning experience; it's a chance to be at the forefront of an emerging field. By the end, attendees will not only have gained valuable skills but also created AI-enhanced features they can take back to their projects or workplaces.Whether you're a seasoned web developer curious about AI or looking to expand your skillset into new and exciting areas, "Hands-On AI: Integrating LangChain with JavaScript for Web Developers" is your gateway to the future of web development. Join us to unlock the potential of AI in your web projects, making them smarter, more interactive, and more engaging for users.
From Todo App to B2B SaaS with Next.js and Clerk
React Summit US 2023React Summit US 2023
153 min
From Todo App to B2B SaaS with Next.js and Clerk
WorkshopFree
Dev Agrawal
Dev Agrawal
If you’re like me, you probably have a million side-project ideas, some that could even make you money as a micro SaaS, or could turn out to be the next billion dollar startup. But how do you know which ones? How do you go from an idea into a functioning product that can be put into the hands of paying customers without quitting your job and sinking all of your time and investment into it? How can your solo side-projects compete with applications built by enormous teams and large enterprise companies?
Building rich SaaS products comes with technical challenges like infrastructure, scaling, availability, security, and complicated subsystems like auth and payments. This is why it’s often the already established tech giants who can reasonably build and operate products like that. However, a new generation of devtools are enabling us developers to easily build complete solutions that take advantage of the best cloud infrastructure available, and offer an experience that allows you to rapidly iterate on your ideas for a low cost of $0. They take all the technical challenges of building and operating software products away from you so that you only have to spend your time building the features that your users want, giving you a reasonable chance to compete against the market by staying incredibly agile and responsive to the needs of users.
In this 3 hour workshop you will start with a simple task management application built with React and Next.js and turn it into a scalable and fully functioning SaaS product by integrating a scalable database (PlanetScale), multi-tenant authentication (Clerk), and subscription based payments (Stripe). You will also learn how the principles of agile software development and domain driven design can help you build products quickly and cost-efficiently, and compete with existing solutions.
Building Pinia From Scratch
Vue.js Live 2024Vue.js Live 2024
70 min
Building Pinia From Scratch
Workshop
Eduardo San Martin Morote
Eduardo San Martin Morote
Let's dive into how Pinia works under the hood by building our own `defineStore()`. During this workshop we will cover some advanced Vue concepts like dependency Injection and effect scopes. It will give you a better understanding of Vue.js Composition API and Pinia. Requirements: experience building applications with Vue and its Composition API.
Mastering 3D Web Development with TresJS ecosystem: A Vue.js Workshop
Vue.js Live 2024Vue.js Live 2024
119 min
Mastering 3D Web Development with TresJS ecosystem: A Vue.js Workshop
Workshop
Alvaro Saburido
Alvaro Saburido
Introducing "Mastering 3D Web Development with TresJS," a specialized workshop crafted for Vue.js developers eager to explore the realm of 3D graphics within their web applications. TresJS, a powerful custom renderer for Vue, is specifically designed to work seamlessly with Vue's reactive system. This workshop offers a deep dive into integrating sophisticated 3D visualizations and interactive experiences directly into Vue applications, leveraging the unique strengths of both Vue and TresJS ecosystems.
This workshop is designed for Vue.js developers looking to expand their skill set into the third dimension, UI/UX designers interested in incorporating 3D elements into web applications, and front-end developers curious about the potential of 3D graphics in enhancing user experiences. You'll need to be familiar with Vue.js to benefit from this workshop fully.
What You Will Learn- Introduction to TresJS: Discover the fundamentals of TresJS and how it integrates with the Vue ecosystem to bring 3D graphics to life.- Creating 3D Scenes with Vue: Learn to construct intricate 3D scenes utilizing Vue components, enhancing your user interfaces with dynamic and immersive visuals.- Interactivity and Animation: Master the techniques to make your 3D scenes interactive, responding to user inputs for a captivating user experience.- Integrating with Vue Features: Explore advanced integration of TresJS with Vue’s reactivity, composables, and the Vuex store to manage state in 3D web applications.- Performance and Best Practices: Gain insights into optimizing your 3D scenes for performance and best practices to maintain smooth, responsive web applications.