Software development is surrounded by a myriad of methods, process models, ecosystems, plugins, languages, architectures, and best practices. On top of that, the entire field is constantly changing: new requirements, new programming languages, new tools. And where can you actually find trained specialists for all these technologies these days? Many factors contribute to the fact that in-house software development can be both expensive and risky.

Depending on the nature of the problem to be solved, there are various approaches. Sometimes it is enough to use purchased software, either on-premise or SaaS, nowadays also often no-code or low-code solutions. For the latter, experience confirms that workload is actually reduced and tasks shift from IT to business departments. This also reduces friction at the interface between IT and business.

AI-Supported Coding Tools as Enablers for In-House Development: A Look at Github Copilot #

There are situations where you need to develop your own software. Good reasons include: no ready-made solution exists for a specific problem, or you expect to gain a significant market advantage with a custom solution. In times of skilled worker shortages, it makes sense to support valuable developer resources and use them as efficiently as possible. Some time ago, Github presented an exciting AI-based optimization solution for the programming process itself in technical preview: the “Github Copilot”.

According to Github, this copilot is “your AI pair programmer”, an artificial intelligence that assists when the programmer is stuck despite the fifth cup of coffee and cannot find an elegant solution for a method bursting with cyclomatic complexity. Technologically, Copilot is based on the AI system Codex developed by OpenAI, the real hero of our story. Trained on publicly available program code and natural language, a powerful AI model has been created that bridges these two types of input. Github Copilot is then an integration layer on top of Codex, adding its functionalities to the existing code completion tooling of an IDE (such as VSCode or IntelliJ).

What Are the Capabilities of Github Copilot? #

Together with a colleague, I recently explored Copilot’s capabilities. What can I say? It can even generate jokes. After the request “Tell me a joke” in a code comment, the following automatic textual suggestion appeared in a console log:

"What did the cheese say when it looked into the mirror?" - "Hello-me (halloumi)"

There are also answers to important questions of our time. The variable isCovid19Real is automatically initialized with true. The same applies to areBirdsGovernmentSpies. Some things are simply true, others make you smile.

However, Github Copilot can do much more. Corresponding implementations can be generated using natural descriptive language in comments or method names. The resulting implementation can even be refined through continuous comments. This works surprisingly well in practice. For example, we generated a method in seconds that outputs successive Fibonacci numbers and adds a string prefix to each number. We also got a clean solution for calculating time differences in a specified time unit for two time formats passed as method parameters. When we changed the object type of one of the parameters, Copilot automatically adapted the implementation. It is also helpful that we usually receive various suggestions for a problem, from which we can choose the most suitable solution. Copilot is not a search engine, but a “code synthesizer”. According to Github’s own study, a large proportion of the generated code is unique relative to the training data. However, this uniqueness has natural limits, as in any rule-based system. This is not a bad thing as long as there is convergence to an optimal solution.

How Useful Is an AI-Supported Coding Tool? #

Github Copilot can at least partially replace the constant browser searching that developers know so well. Some of those perpetually open Stackoverflow tabs can now be closed for good. Stackoverflow and other discussion pages are not only a natural source of solutions, but also a real source of explanations. So if you want to understand why a solution might be good and useful, you still need to engage with other real programmers. For experienced developers, Copilot is helpful as an intelligent code completion tool that quickly generates initial implementation suggestions for entire methods or specific problems in day-to-day work. For less experienced developers, Copilot at least provides starting points whose usefulness should be questioned afterwards. The greatest danger when using such a tool is blind application. If suggestions are simply adopted into a new code base without questioning their usefulness for the specific problem, this can result in bad code: bugs, poor performance, and technical debt.

What Are the Limitations of AI-Supported Programming? #

At its core, programming is a creative process. Within a system with certain rules (a programming language, frameworks, etc.), solutions must be found for known and sometimes unknown problems. These solutions should be robust, optimized, and reliable. The relationship to the rest of the code base and architecture also plays a role.

Artificial intelligence is always based on data. This data must be generated beforehand in one way or another. For programming languages such as Java, C++, and Python, there is a large amount of publicly accessible, high-quality code written by various programmers for a wide variety of situations. The AI’s suggestions will be correspondingly good. But even then, suggestions should be taken with a grain of salt. Github’s own benchmark reveals that currently only around 43% of implementation suggestions are correct on the first attempt. There is also a risk of reproducing unsafe coding patterns, inserting bugs, or using outdated APIs. Github itself writes:

"There's a lot of public code in the world with insecure coding patterns,
bugs, or references to outdated APIs or idioms.
When GitHub Copilot synthesizes code suggestions based on this data,
it can also synthesize code that contains these undesirable patterns."

It follows that when new programming languages are used, the AI will have greater difficulty generating good suggestions. But the devil is in the detail. Many programming languages are similar, and AI can benefit from a broad data basis.

What Other Tools Are Available for AI-Supported Programming? #

It is not just Github, a platform owned by Microsoft, that is working on AI-supported programming tools. Providers such as Tabnine, Kite, and Sourcery pursue similar approaches. Tabnine and Kite offer support for various programming languages, while Sourcery is limited to Python. However, none of these tools go as far as Github Copilot or Codex, which allow code creation based solely on natural language. Github Copilot was preceded by Microsoft initiatives such as DeepCoder and the PROSE research project. Other tech companies are also focusing on AI-supported programming. Google uses AI capabilities in its AutoML tool to create better AI systems and give less experienced developers the opportunity to perform machine learning. Amazon developed the AI tool CodeGuru to give developers the opportunity for intelligent code review.

Will AI Make Programmers Superfluous in the Future? #

In view of this rapid development in AI-supported programming, future enthusiasts naturally wonder whether an AI system will be able to replace programmers in the foreseeable future. At present, the answer is a clear “no”.

For software solutions, purchased, low-code, and no-code solutions will probably prevail in the medium term wherever there is nothing original or essentially innovative to implement. These solutions share the characteristic that the same code is used for the same problems, optimized thoroughly by the provider. In such situations, the usefulness of AI programming approaches is limited for now.

For in-house development, the choice of technology, planned architecture, and complexity of underlying functions influence the options for using an AI programmer. However, even under optimal conditions, the generated code can only serve as support. It is not error-free and cannot be automatically integrated into the complex context of an application landscape without further work.

Today, AI systems can be used to increase developer efficiency in daily work. Promoting the use of modern systems therefore has a direct impact on in-house development projects. To put it bluntly: this may even be a way to temporarily alleviate the shortage of skilled workers. However, completely replacing programmers with artificial intelligence is not yet possible. Only a human programmer can currently implement program code based on complex specifications and requirements, link a multi-layered tech stack, or solve difficult problems. We can also expect that the role of the software developer will not disappear, but simply change. Instead of writing code themselves, software developers may primarily curate data on which an AI efficiently writes code. Perhaps we will all end up becoming data scientists.

Please note that this article was originally published with a former employer and is available here in its original German version.