English
Prompt engineering and AI are booming, with almost every startup team integrating AI to make things easier for their users.
Today, we are covering 17 projects to maximize your productivity for developers building with AI.
You will find tools related to prompt engineering, code editors, agents and many more exciting things.
This list will surprise you.
1. Latitude LLM – prompt engineering platform to build and refine prompts with AI.
Latitude is the open source prompt engineering platform to build, evaluate, and refine your prompts with AI. You can create and iterate prompts in the platform by using their SDKs or the API.
The best part is that every time a prompt runs, it automatically logs the entire context, the output and other metadata relevant for evaluations and debugging.
This is how the dashboard looks.
✅ There is support for advanced features like parameters, snippets, logic and more.
✅ You get version control for prompts, collaborative prompt manager and even evaluations in batch or real-time.
A basic user flow can be:
-→ Create a new project.
-→ Write your first prompt using the editor.
-→ Test your prompt using the playground with different inputs and see the mode’s responses.
-→ Before deploying, you can upload a dataset and run a batch evaluation to assess your prompt’s performance across various scenarios. Watch this video to see how evaluations can analyze the results of your prompts.
-→ You can deploy your prompt as an endpoint for easy integration with your applications.
-→ Use the Logs section to review your prompt’s performance over time.
-→ Refine your prompt and invite team members to your Latitude workspace to collaborate.
Watch this quick demo to learn more.
You can read the docs and the concepts involved like prompts, logs and evaluation that are involved.
You can use this quickstart guide by using the cloud version or self-hosting it.
They have 536 stars on GitHub and are growing very fast.
2. LiveKit Agents – build real-time multimodal AI apps.
LiveKit Agents is an end-to-end framework that enables developers to build intelligent, multimodal voice assistants (AI agents) capable of engaging users through voice, video and data channels.
Let me explain in simple words.
The Agents framework allows you to build AI-driven server programs that can see, hear and speak in real time. Your agent connects with end-user devices through a LiveKit session. During that session, your agent can process text, audio, images or video streaming from a user’s device and have an AI model generate any combination of those same modalities as output and stream them back to the user.
✅ They support a lot of SDKs including Swift, Android, Flutter, Rust, Unity, Node, Go, PHP, React and more.
You can get started with pip.
pip install livekit-agents
They also have a lot of plugins that make it easy to process streaming input or generate output. For instance, there are plugins for converting text-to-speech or running inference with popular LLMs. One example of a plugin is:
pip install livekit-plugins-openai
They also provide open source React components and examples for building with LiveKit if you’re using React.
You can read the docs and see the list of all plugins that are available. If you want to try, then you can do it at cloud.livekit.io.
If you’re looking for some sample apps with code, check these:
⚡ A basic voice agent using a pipeline of STT, LLM, and TTS
⚡ Super fast voice agent using Cerebras hosted Llama 3.1
This is one of the most exciting projects out of 1000+ projects that I have ever seen in open source.
They have 3.2k stars on GitHub and are growing strong.
3. Julep – build stateful AI apps.
Julep is a platform for creating AI agents that remember past interactions and can perform complex tasks.
Imagine you want to build an AI agent that can do more than just answer simple questions. It needs to handle complex tasks, remember past interactions and maybe even use other tools or APIs. That’s exactly where Julep comes in.
You can use it to create multi-step tasks incorporating decision-making, loops, parallel processing and a whole lot more.
✅ It will automatically retry failed steps, resend messages, and generally keep your tasks running smoothly.
✅ You can use Julep’s document store to build a system for retrieving and using your own data.
To get started, you can use npm or pip.
npm install @julep/sdk
or
pip install julep
There is also a quick example that I recommend reading where a sample Agent selects a topic, generates 100 related search queries, performs the searches simultaneously, summarizes the results and shares the summary on Discord. With proper code 🙂
Watch this quick demo and check more examples apps to understand more.
You can read the detailed docs which has a Python quickstart guide, Nodejs quickstart guide, tutorials and how-to guides.
You might say it’s similar to Langchain but they both have slightly different concepts. For instance, LangChain is great for creating sequences of prompts and managing interactions with AI models. It has a large ecosystem with lots of pre-built integrations, which makes it easier to run things quickly.
Julep on the other hand is more about building persistent AI agents that can remember things. It shines when you need complex tasks that involve multiple steps, decision-making, and integration with various tools or APIs directly within the agent’s process. Read more on the detailed comparison.
Julep has 1.3k stars on GitHub and growing strong.
4. Open WebUI – most loved AI Interface (Supports Ollama, OpenAI API…), runs offline.
Open WebUI is an awesome user-friendly self-hosted chat user interface designed to operate entirely offline.
This can help you build features at a rate you can never imagine.
You can use pip to quickly install it. Check the complete installation guide.
# install Open WebUI
pip install open-webui
# run Open WebUI
open-webui serve
Let’s see some of the awesome features.
✅ You can customize the OpenAI API URL to link with LMStudio, GroqCloud, Mistral, OpenRouter and more.
✅ You can use it in your preferred language with our internationalization (i18n) support.
✅ There is an option of hands-free voice and video call features which gives a little more flexibility.
✅ Their official website has clear info on a bunch of models, prompts, tools and functions by the community.
✅ You can load documents directly into the chat or add files to your document library and access them using the # command before a query.
✅ You can perform web searches using providers like SearXNG, Google PSE, Brave Search, serpstack, serper, Serply, DuckDuckGo, TavilySearch and SearchApi to inject the results directly into your chat experience.
Also recommend watching this walkthrough to learn more.
You can read the docs which includes a getting started guide, FAQs (recommend reading) and tutorials.
It is built using Svelte, Python and TypeScript.
They have 41.6k stars on GitHub which says a lot about the popularity.
5. Quivr – RAG framework for building GenAI second brains.
Quivr, your second brain, utilizes the power of GenerativeAI to be your personal assistant. You can think of it as Obsidian but turbocharged with AI powers.
It is a platform that helps you create AI assistants, referred to as Brain. These assistants are designed with specialized cases like some can connect to specific data sources, allowing users to interact directly with the data.
While others serve as specialized tools for particular use cases, powered by Rag technology. These tools process specific inputs to generate practical outputs, such as summaries, translations and more.
Watch a quick demo of Quivr!
Some of the amazing features are:
✅ You can choose the type of Brain you want to use, based on the data source you wish to interact with.
✅ They also provide a powerful feature to share your brain with others. This can be done by sharing with individuals via their emails and assigning them specific rights.
✅ Quivr works offline, so you can access your data anytime, anywhere.
✅ You can access and continue your past conversations with your brains.
✅ But the best one that I loved is that you can install a Slack bot. Refer to this demo to see what you can do. Very cool!
Anyway, read about all the awesome stuff that you can do with Quivr.
You can read the installation guide and 60 seconds installation video. Refer to the docs for any other information.
They have also provided guides on how to deploy Quivr with Vercel, Porter, AWS and Digital Ocean.
It has 36.3k+ Stars on GitHub with 300+ releases.
6. Dify – innovation engine for GenAI apps.
Dify is an open-source platform for building AI applications.
Its intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
They combine Backend-as-a-Service and LLMOps to improve the development of generative AI solutions. You can use the cloud or self-host it (refer to docs).
You can even build and test powerful AI workflows on a visual canvas.
Let’s see some of the awesome features:
✅ Dify provides 50+ built-in tools for AI agents, such as Google Search, DALL·E, Stable Diffusion and WolframAlpha.
✅ You can monitor and analyze application logs and performance over time.
✅ You can use the RAG pipeline to extract text from PDFs, PPTs and other common document formats.
✅ A lot of integration options are available from dozens of inference providers and self-hosted solutions, covering GPT, Mistral, Llama3, and any OpenAI API-compatible models. A full list of supported model providers can be found here.
✅ You can create AI Agents with just a few clicks, letting them independently use enterprise-defined tools and data to solve complex tasks.
You can read the docs.
Two of the impressive use cases that I loved:
⚡ Building a Notion AI Assistant
⚡ Create a MidJourney Prompt Bot with Dify
Dify has 47.7k stars on GitHub and has a lot of contributors.
7. Micro Agent – AI agent that writes (actually useful) code for you.
AI-assisted coding tools like GitHub Copilot and ChatGPT don’t produce very reliable code and they often don’t work correctly right out of the box, you find bugs, edge cases, or even references to non-existent APIs.
This can lead to a frustrating loop of trying the generated code, finding issues, going back to the AI for fixes and repeating.
The time spent debugging can negate the total time saved using AI tools in the first place.
Micro Agent uses AI to mitigate the problems of unreliable code generation.
Give it a prompt and it’ll generate a test and then iterate on code until all test cases pass.
You can install it using this command.
npm install -g @builder.io/micro-agent
# Next, set your OpenAI API key when prompted or manually using this.
micro-agent config set OPENAI_KEY=<your token>
# Then you can run to start a new coding task
micro-agent
Micro Agent will prompt you to describe the function you want, generate tests, and start writing code in your preferred language to make the tests pass. Once all the tests are green, you’ll have a fully functional, test-backed function ready to use.
Let’s explore some of the most mind blowing use cases:
⚡ 30-second demo of Micro Agent generating tests and code for a TypeScript function that groups anagrams together from an array of strings.
⚡ Using Micro Agent to generate a simple HTML to AST parser (it was achieved on two iterations).
⚡ Unit test matching.
⚡ Visual matching (experimental).
⚡ Integration with Figma.
Micro Agent can also integrate with Visual Copilot to connect directly with Figma to ensure the highest fidelity possible design to code!
Visual Copilot connects directly to Figma to assist with pixel-perfect conversion, exact design token mapping, and precise usage of your components in the generated output.
Then, Micro Agent can take the output of Visual Copilot and make final adjustments to the code to ensure it passes TSC, lint, tests, and fully matches your design including final tweaks. Amazing right 🙂
You can read the docs and the official blog where the team discussed everything about the micro agent.
It’s open source with 2.8k stars on GitHub.
8. Cline – autonomous coding agent right in your IDE.
The concept seems very similar to Cursor where Cline is an autonomous coding agent capable of creating/editing files, executing commands, and more with your permission every step of the way.
It’s a VSCode extension and you can find it in the marketplace. It has 84k+ installs.
Cline works on Claude 3.5 Sonnet’s agentic coding capabilities.
✅ Cline supports API providers like OpenRouter, Anthropic, OpenAI, Google Gemini, AWS Bedrock, Azure, and GCP Vertex. You can also configure any OpenAI compatible API or use a local model through Ollama.
✅ You can add context using four different commands.
@url: Paste in a URL for the extension to fetch and convert to markdown, useful when you want to give Cline the latest docs@problems: Add workspace errors and warnings (‘Problems’ panel) for Cline to fix.
✅ It uses a headless browser to inspect any website, like localhost, allowing it to capture screenshots and console logs. This gives him the autonomy to fix visual bugs and runtime issues without you needing to handhold and copy-pasting error logs yourself.
✅ You can even run commands in the terminal to do awesome stuff.
You can read the docs.
Cline has 7k stars on GitHub.
9. GPT Crawler – create your own custom GPT from a URL.
With GPT Crawler, you can crawl any site to generate knowledge files to make your own custom GPT from one or multiple URLs.
The objective is to make the docs site interactive, people can more simply find the answers they are looking for using a chat interface.
Watch this quick demo!
You will have to configure the crawler and then simply run it. After the crawl is complete, you will have a new output.json file, which includes the title, URL and extracted text from all the crawled pages.
You can now upload this directly to ChatGPT by creating a new GPT. Once uploaded, this GPT assistant will have all the information from those docs and be able to answer unlimited questions about them.
It’s officially a assistant in ChatGPT.
You can read the docs on how to get started. You can find all the instructions on the official blog.
If you are wondering how Mitosis compiles those components, then watch this quick tutorial.
They have 18.6k stars on GitHub.
10. Composio – production ready toolset for AI Agents.
Composio is the only tool needed to build complex AI automation software. It allows AI models to access third-party tools and applications to automate their interactions with them.
For instance, you can connect GitHub with the GPT model via Composio and automate reviewing PRs, resolving issues, writing test cases and more.
You can automate complex real-world workflows by using 90+ tools and integration options such as GitHub, Jira, Slack and Gmail.
You can also automate actions like sending emails, simulating clicks, placing orders and much more just by adding the OpenAPI spec of your apps to Composio.
This is how you can use this.
# install it
pip install composio-core
# Add a GitHub integration
composio add github
Here is how you can use the GitHub integration to star a repository.
from openai import OpenAI
from composio_openai import ComposioToolSet, App
openai_client = OpenAI(api_key="******OPENAIKEY******")
# Initialise the Composio Tool Set
composio_toolset = ComposioToolSet(api_key="**\\*\\***COMPOSIO_API_KEY**\\*\\***")
## Step 4
# Get GitHub tools that are pre-configured
actions = composio_toolset.get_actions(actions=[Action.GITHUB_ACTIVITY_STAR_REPO_FOR_AUTHENTICATED_USER])
## Step 5
my_task = "Star a repo ComposioHQ/composio on GitHub"
# Create a chat completion request to decide on the action
response = openai_client.chat.completions.create(
model="gpt-4-turbo",
tools=actions, # Passing actions we fetched earlier.
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": my_task}
]
)
You can read the docs and examples.
Composio has 9k stars on GitHub.