How do you iterate on and manage prompts in production?
Ivan Pashchenko
5 replies
I'm curious about how people handle managing and iterating on their prompts, especially for production-facing prompts that many users depend on.
- Where do you store your prompts?
- Do you use version control?
- How do you test prompts after editing?
- Where do you store your test sets?
- Do you evaluate results? If so, how?
- Are you fine-tuning models like GPT-3.5 for better/cheaper results?
Replies
Sol Yousefi ☀️@solyou
Contentify AI provides a centralized platform for storing, editing, and testing prompts. With version control capabilities, users can track changes and easily revert if needed. Prompt testing can be done within the platform, with test sets stored conveniently for evaluation. Additionally, Contentify AI allows for fine-tuning models like GPT-3.5 to continuously improve prompt generation for better results.
Problem: Managing and iterating on prompts for production can be time-consuming and complex, often requiring multiple tools and processes. Contentify AI streamlines this by offering a one-stop solution for prompt management, editing, testing, and evaluation. With AI-powered agents, users can easily generate, optimize, and schedule prompts across various platforms, saving time and improving efficiency.
Try Contentify AI for Free here just look up Contentify AI
Share
@ivan_pashchenko interesting. Out of curiosity, what is the use case?
@ivan_pashchenko Can you give me an example?
Managing and iterating on production prompts is like fine-tuning a recipe: store prompts in a secure database "pantry," use version control as your virtual sous chef, and taste-test with stored test sets to ensure it's not a coding catastrophe! And remember, evaluating results is key - it's like being a prompt detective, hunting for the perfect response clues! 🕵️♂️
Wrte (pre-launch)
@thestarkster so where do you store prompts and tests? DB? How do you version it then? How do you run your tests when prompt has been changed?