At Tabnine, we strongly believe that the future of software development is AI-driven, where AI engineers (agents) collaborate closely with human engineers to complete a task and AI orchestrates all aspects of the software development life cycle.
We’re actively building for this future, and today we’re offering a glimpse of a fully autonomous version of Tabnine — an AI agent that can implement a Jira ticket in the context of an existing project and create a fully functioning application. It’s early days for Tabnine’s “Jira-to-code” AI agent, but we’re excited about how this can truly transform the software development processes.
In the demo below, the Tabnine AI agent takes simple specifications stored in Atlassian Jira and creates a fully functioning application:
Real-world requirements for an “AI engineer”
Recently announced agents such as SWE-Agent, gpt-engineer, Devika, and Devin show great promise and demonstrate the increasing potential of AI agents in the software development lifecycle. Agents are equipped with an extensive toolbelt (similar to the tools available to developers) to complete their tasks. To make such agents a productive reality in an enterprise setting, we believe that agents should have these properties:
Concise: Communication with the agent must be concise. This is true both for providing input to the agent as well as for consuming its output. A major part of software development is discovering the desired behavior from a typically underspecified definition. The effectiveness of an agent greatly depends on the ability to iterate on a task quickly with compact input and output.
Personalized and incremental: The agent must be aware of the current codebase, the organization’s best practices, documentation, and the style and preference of the human developer it assists. An agent that generates code foreign to the current codebase, potentially regenerating instead of reusing components, limits its applicability.
Reliable: The human must trust the output the agent provides. Reviewing the thousands of lines of code an agent generates to find subtle bugs might limit the value of delegating tasks to the agent.
Inquisitive: An effective agent must be able to probe the human for additional context and instructions where needed. The efficacy of the human <-> AI dialog is critical for the agent’s success in real life.
Self-aware (reflective): The agent must recognize when it has completed a task with a nonsatisfactory result. An effective agent should be able to come back to the human with feedback on how the task was completed in a way that could guide its improvement process.
Tabnine’s approach to making AI agents a reality
Here’s how we’re addressing the challenges that prevent AI coding assistants from being fully embedded in the software development life cycle:
To facilitate concise communication, we deliver the AI agent via Tabnine Chat — an enterprise-grade, code-centric chat application that allows developers to interact with Tabnine AI models using natural language. This is the natural place where developers work and it allows them to relate to a Jira ticket in the context of the existing project and even in the context of an existing module/file of the project. It gives the developer fine-grained control over how the ticket is implemented by and lets them manually modify the generated code through conversing with the agent. Further, it allows the agent to ask clarifying questions and probe for additional input from the developer as part of being “inquisitive.” Of course, not all agents would complete their work interactively, but you can keep long-running agents running in the background, similar to running your tests.
As a foundational phase, we focus on personalization, exposing the agent to the wealth of information available about the project in Tabnine and providing the agent with tools exposed by Tabnine supporting in-IDE operations. This involves retrieval from Tabnine’s vector memory (based on embeddings) and Tabnine’s semantic memory (based on relationships between semantic entities).
A sandboxed execution environment allows Tabnine agents to execute generated code and check its adherence to test cases (both human-provided and generated ones). We plan to improve this step further by integrating additional tools.
Tabnine agents ask humans clarifying questions and follow-up questions. This is a key element of our approach, and we believe it’s a mandatory aspect of an agent facing any task with realistic complexity.
Agents are asked to reflect on their work. This reflection may result in internal adjustments to the plan or additional questions to the human developer (“human in the loop”).
What’s coming next?
Tabnine’s Jira-to-code AI agent is still in the works and we’re experimenting internally to improve its abilities to support the aforementioned challenges. We’ll gradually release the ability to contextualize all Tabnine agents on information from Jira. Stay tuned.
