What GitHub’s Copilot Tool Reveals About the Future of AI in Software Development


Software developers speak the language of computers. Knowing the commands and symbols, engineers rely on their coding skills to build applications.

The tools that support developers are evolving, making the next generation of engineers closer to conductors who rely on algorithms to turn natural language signals into applications. With AI feedback, the tools promise software applications that assemble quickly and easily.

This is the essence of Co-pilot, a tool built by GitHub and OpenAI. Essentially, the platform is an AI-infused wizard that works like a virtual pair programmer for developers. Formed by code available on public repositories, Copilot is currently available in technical preview.

GitHub Copilot uses OpenAI Codex, an engine based on GPT-3, which “pulls context from comments and code, and instantly suggests individual lines and entire functions,” according to the company’s website.

While not the only tool promising an AI-assisted software development process, it is potentially a step forward as it can take advantage of the machine learning capabilities of GPT-3 to convert the natural language into code, according to Arun Batchu, senior research director at Gartner. In a way, this can facilitate the disconnect between the requirements of the business and the software that is being built.

“This is what excites me about the potential of the new generation of tools,” said Batchu. “They are raising the bar.”

With smart tools complementing the development, the software developer of the future will focus more on the connection between business requirements and technology. But the tools that herald this change have yet to reach the level of maturity required for widespread use and trust. They must first ensure their security before wide adoption at the enterprise level.

AI, the dream pair programmer?

In the growing market for developer tools providers, there were previously a number of solutions that performed the same functions as GitHub. Vendors like Tabnine, Kite, and Codota can generate suggestions in real time, using open source repositories and libraries.

But where other code completion tools allowed rapid prototyping or the creation of “skeleton” applications that developers then fleshed out, Copilot aims to take that process of fleshing out further, said Usman Lakhani, research director at Info-Tech Research Group.

“This is where the Copilot tool tries to find its way,” Lakhani said.

According to GitHub, although Copilot is trained on existing code libraries, “the vast majority of the code it suggests is uniquely generated and has never been seen before.” The company declined an interview request, but asked the product owner the questions. FAQ section.


“I still think most companies … overall haven’t seen a significant transformation in the software development lifecycle with AI.”

Nitzan Mekel-Bobrov

Head of AI at eBay


Another market difference with Copilot is the adoption of GPT-3. This technology, which in the past wrote an article describing himself, is orders of magnitude above its predecessor, GPT-2. Where GPT-2 was able to rely on 1.5 billion parameters to produce its results, the new model can exploit 175 billion parameters.

“What is happening from an AI perspective, and in particular the new generation of deep learning based AI, just as it elevates or transforms all other jobs, it is clearly transforming the work of creation. software itself, ”Batchu said.

AI has spared few of the traditional roles of redefining. In the supply chain space, automation reframed how warehouse workers work. Bots have entered the marketing arena, suggesting examples of effective marketing copy. AI has also responded to many customer service requests that were once handled by humans.

Tools like GitHub’s Copilot use AI to help engineers “figure out what shape the app you need to build, but it won’t be the perfect one,” Batchu said.

“Copilot guesses the exact code I want to write about once in ten, and the rest of the time, it suggests something that’s pretty good, or completely wrong”, François Zaninotto, CEO and co-founder of Marmelab, commented in a Hacker News article. “But when he guesses correctly, I feel like he’s reading my mind.”

Is the technology still there?

Copilot is part of a larger trend: the potential of AI as a generative tool, as highlighted by Gartner.

Rather than sit on the sidelines and help, AI can produce on its own, whether this product is more effective marketing copy, news routes for product delivery, New chemical compounds with therapeutic potential or, as Copilot does, the application infrastructure and lines of code that power it.

“I still think that most companies, certainly most large companies but probably also smaller ones, overall, have yet to see a significant transformation in the software development lifecycle with AI.” said Nitzan Mekel-Bobrov, director of AI at eBay. .

Although versions of these applications already exist, the industry is still evaluating the long-term impact of AI on the software development process.

“Where I see the initial applications, or the first phase of it, is really applying AI to the boundaries of the existing process, rather than changing the process as a whole,” Mekel-Bobrov said.


“What is happening from an AI perspective, and in particular the new generation of deep learning based AI, just as it elevates or transforms all other jobs, it is clearly transforming the work of creation. software itself. “

Arun Batchu

Senior Research Director at Gartner


When it comes to code output, the question arises as to whether an algorithm is capable of applying engineering best practices when producing code on its own.

“I felt like, I was sifting through, trying to find that needle in the haystack, initially,” said Lakhani, who had access to the technical review version of the product and found that most of the code did not meet coding best practices. . “However, that doesn’t mean it won’t get better.”

Security will be another factor executives will want to consider if AI pair programmers become widely adopted. Insecure code heading to an enterprise application is a source of problems.

One of the biggest risks with auto-generated code is that “they may adhere to nine out of ten best practices, but the best practice it lacks is one that will put a business at the greatest risk,” Lakhani said.

In one study, researchers gave Copilot 89 different scenarios to complete and produced 1,692 programs. Upon examination, approximately 40% of the code contained cybersecurity vulnerabilities, according to researchers Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri.

But some believe that AI tools can actually make software development more secure. According to Dori Exterman, CTO at Incredibuild, using verified code databases to create software could reduce the chances of unintentionally adding security vulnerabilities to an application through freshly written internal code.

“As a developer, I’m more limited, in a way, in understanding the security vulnerabilities that the code I generate can create,” Exterman said.

In addition to security concerns, there are questions about the flexibility of the platforms and how efficiently they can be integrated into the existing coding process, given their high demand for IT resources.

“The problem with Copilot is that you still need advanced AI hardware, because the model is so big and so compute-intensive,” Batchu said.

The future of programmers

What does it mean to be a software developer? The answer has evolved with the tools of the trade.

“The trend, so far, is a bit similar in a way,” Exterman said. “All software development ecosystems try to minimize the amount of coding work that developers have to do. It started, for example, with libraries 30 years ago, and more.”


“Copilot guesses the exact code I want to write about once in ten, and the rest of the time it suggests something pretty good, or completely wrong.”

Francois Zaninotto

CEO and co-founder of Marmelab


In a few decades, as AI-assisted tools mature, their addition to the software development process could have lasting impacts on the profession. Experts see that a new kind of software developer profile will emerge with more help from AI.

“I really think it’s a move to a more generalist than a specialist,” Mekel-Bobrov said. “The specific language requirements will relax or disappear altogether. “

AI platforms like GitHub Copilot, when ready for enterprise-grade deployment, will cover the ground between human specification and code execution. This has an impact on how businesses view software engineers and the role they can play for businesses.

“I think this will allow software developers to act more as software designers, for lack of a better term, rather than focusing more on implementation,” Mekel-Bobrov said. “This would allow them to spend more time designing and building the software development process, and less, if at all, on routine tasks.”

With AI tools capable of converting natural language into snippets, or even entire applications, businesses can iterate faster. From the perspective of enterprise business units that look to IT for solutions, software developers will evolve into software problem solvers, according to Batchu.

“In many ways, the software craft is going to start to happen,” Batchu said.


Gordon K. Morehouse