Blog
AI Agent is approaching to production ready by leverage emerging managed infrastructure

AI Agent is approaching to production ready by leverage emerging managed infrastructure

Wells Wang<w@npi.ai> • 2024/06/10

before-arch

vs

after-arch

Madrona Venture, an investor in companies like Amazon, Snowflake, and UiPath, posted an insightful article The Rise of AI Agent Infrastructure (opens in a new tab) by Jon Turow on June 5th, detailing the latest trends in AI agent development.

As an AI Agent developer, I found this article to be the one of best I’ve read in 2024, echoing many of my own views. I strongly recommend anyone involved with AI agents spend 10 minutes reading it.

Here are some highlights from my personal perspective and understanding of the article:

  1. AI agent workloads are shifting from self-hosted to managed services.

  2. More specialized AI agent development tools are emerging, helping developers build more robust agents.

These trends are familiar to us all. About fifteen years ago, we began migrating our workloads from private data centers to the cloud, embracing various cloud-native development tools like S3, SQS, RDS, Redis, MongoDB, Snowflake, Okta, and more, instead of maintaining our own infrastructure.

From Ed Huang, the co-founder and CTO of PingCAP, I learned a valuable software engineering philosophy: "Make it work, then make it right, and finally, make it fast." I believe AI agents will follow a similar process.

In the past 18 months, we are struggled with building reliable AI Agent for lacking best practices, and no specified products. Thus, we had to build everything from scratch to make AI Agents work. Products like LangChain were the best choice of this stage.

As we're exploring deeper, we try to make AI agents work right and faster, which involving more code tasks to accomplish. As a result, this led to the common patterns are rapidly recognized by the industry: Fine-tuning, RAG, Evaluation, multi-agent collaboration, and tool use(also known as function calling). But each of these patterns are packed with technical details and are evolving quickly, making it challenging to keep up with the latest advancements. Attempting to maintain all this infrastructure would be a massive technical overhead and financial cost.

At this time, we naturally recall our past successes story: we desire more composable specific tools within best practices. Integrating these tools into our products then focus on business-critical value, enabling us to build a product that satisfies our users quickly and effectively.

Therefore, we witness the emergence of a flourishing ecosystem of specialized tools. Each tool will focus on addressing specific problems in AI agent development, replacing the previous all-in-one solutions, giving developers more space and flexibility to make AI agents work right and faster.

Otherwise, I would like to select 4 high-potential open-source tools from Madrona Venture's landscape you may interest, There are:

NPi - Function calling API platform

E2B.dev - Secure sandboxes for AI-generated code execution

CrewAI - Multiagents Orchestration Framework

Agentops - Agent observability toolkits

If you enjoy with this post and would like to discuss more, you could find me by Email, X.com (opens in a new tab), or LinkedIn (opens in a new tab).

Understanding Function Calling in LLMs and Its Difference to RAGHow to use NPi building an AI Agent for meeting scheduling