Part One – Hey ChatGPT, Can You Write Me a Blog Post About Prompt Engineering?

Emily von Bergen
November 8, 2024
minute read

Large Language Models (LLMs) burst onto the scene a few years ago, promising to revolutionize the way we work. While their emergence has certainly transformed many industries, it has also created a demand for new skill sets—one of the most important being prompt engineering.

Prompt engineering involves the skillful structuring of instructions to guide generative Artificial Intelligence (AI) models in producing the desired outcomes, blending both technical precision and creativity.

By structuring prompts effectively, it enables end-users to obtain accurate, high-quality, customized, and relevant responses, avoiding wrong, fabricated, or nonsensical responses. With today’s more advanced models, it has become easier for models to interpret user intent with minimal input, becoming more user-friendly, and in exchange, users have adapted their language to be better understood by LLMs. An example of this can be found in the image below.

In a typical conversation between an end-user and a chatbot, prompt engineering involves crafting specific questions or instructions that guide the chatbot’s responses, ensuring clarity and relevance. These prompts are then sent to the language model for processing, and the responses of the LLM are sent back via the API.

Naturally, mastering the skills for prompt engineering remains challenging for newcomers. Not getting the right output can be frustrating. This often stems from not asking the right questions or providing the right prompt. In this blog post, you will learn how to master the art of crafting precise and effective prompts through practice and experimentation. We’ll cover a lot, from the basics to advanced concepts like using tree-of-thought techniques in a prompt.

This blog builds on ideas begun in another post: How to Build Smarter Apps with Function Calling & Generative AI. We highly recommend you read it first, especially if you are not familiar with Function Calling.

Different prompt types

To enhance your understanding of how AI models operate, it is crucial to understand the distinct types of prompts involved in the interaction process. As illustrated in the image above, both the system prompt and the user prompt are sent to the language model via API calls. These prompts serve different roles in guiding the AI’s responses.

System Prompt

The system prompt represents the foundational behavior and guidelines of the AI model. It gives the model, for example, a role, tone, ethical regulations, and/or subject specification. This is set by a prompt engineer – or you, after reading this blog post. An example of this type of prompt is: “You are a helpful assistant who provides only information about Mendix” – providing direct guidelines on the role, content, and limitations. As you read on, you will find additional and more advanced samples.

User Prompt

A user prompt is another fundamental type. It is the user’s input, question, or request sent to the LLM as portrayed in the image shared earlier in this post. This is the place where end-users can write anything they please, such as “Give me 5 ideas to create a cool App in Mendix”. Of course, for more complex inquiries, end-users can use this post to improve their prompt skills and get more reliable or fitting results from LLMs.

Context Prompt

Depending on the project or use case, adding contextual information to the model might be needed or required. Normally, this information, called context prompt or conversation history, is sent in the same interaction as the system and user prompt. It captures the historical information of the conversation to maintain coherence with the end user and be context aware. This is set by developers within their application, for example, in Mendix, it is captured through a microflow using the Chat Completion (with history) operation.

To understand this concept, imagine a user interacting with a chatbot while asking, “How should I start?” If in previous interactions, the user asked about Mendix, the LLM will understand that the question refers to Mendix apps. In cases where the context is not needed, such as in command-based interactions where the inquiry could be: “Turn on the lights” and the LLM does not need any historical conversation, developers can use operations like Chat Completion (without history).

Essential best practices for prompt creation

When onboarding a new team member, you want to make sure that the task you are giving them will be performed in the most efficient and fastest way. This also applies to prompt development. Now that you know a few types and techniques of prompts, it is crucial to understand the essential best practices for prompt creation, such as:

Be clear: Make sure that the prompt contains relevant details emphasizing a clear and precise message for the LLM to understand. Avoid complex language. An example could be changing sentences like “Evaluate procedural methodologies…,” which is more formal academic writing, to “Explain the process…”.
Describe the context: Providing a background story, scenario, or explanation to the LLM will typically result in a more accurate result due to the context establishment. For instance, if you want your end users to learn about Mendix, the prompt should include this detail.
Provide details and limitations: Specifying details or limitations in the prompt provides more concrete and fitting responses to your use case. These can include instructions – for instance, requesting the LLM to include sources or examples in each response – or adjust how the outcome should look like, such as limiting the answer to 100 words or writing the output in a JSON format.
Let the LLM assume a role: Providing the LLM with a persona and some context on its role can personalize the model to fit your use case better. For example, you build an app that teaches about data science. Then, the prompt should include the role of a professor whose audience is new to data science.
Refine your prompt and work iteratively: Testing and improving the prompt is a crucial practice. Sometimes, even minor changes, such as in the model configuration, i.e., from GPT3.5-Turbo to GPT4o-mini, could cause significant differences in the LLM behavior. Documenting these changes and the outcomes can help find the optimal prompt for your use case. Optionally and if available, having other people testing the prompt for feedback can provide another form of iterative refinement.

Top 5 prompt techniques

The more complex a use case is, the more complex the prompt might become. However, do not worry if the initial outcome is not immediately what is desired. Rather, elevate your prompt skills by using one of the following prompt techniques:

Prompt technique	What it is	Use case and prompt example
Interview Pattern Approach	Using an interview-style approach allows the model to ask follow-up questions to the end-user to provide a better fitting response.	Use case: Movie Recommendation Engine. Prompt instruction: “You will act as a movie recommender expert. You will ask the user a series of detailed questions, one at a time, to understand their preferences in movies.”
Instruction Prompt	The instruction gives the AI model directions on how to perform a task. It can be a guidance on the type of output, such as summarization or translation, style, format, and more.	Use case: Mendix ML Kit Python Script Generator Prompt instruction: “You will act as an expert Python developer specializing in the Mendix ML Kit. The output/response should be given in a python script with annotations for the Mendix ML Kit.”
Few-shot Prompt	It helps the model to learn a task or pattern dynamically by providing examples. It can also be part of the system prompt.	Use case: English-Spanish Translator Prompt instruction: “You are a kind assistant that helps translate English texts to Spanish. For example, ‘Good Evening’ to Spanish: ‘Buenas Noches'”
Chain-of-Thought	It simplifies complex tasks by turning them into discrete steps that happen in a certain order.	Use case: Medical Diagnosis for interns. Prompt instruction: “You are a diagnosis assistant designed to help trainee doctors ask a series of questions for patients’ initial evaluation. Your goal is to identify the patient’s symptoms, health history, and other relevant variables to reach an accurate evaluation that, depending on the result, will be forwarded to nurses or doctors. Start by asking the patient about their primary symptoms and the reason for their visit. Then, …”
Tree-of-Thought	Similar to a decision tree, it includes several lines of thought to allow the model to evaluate and find its path to the correct outcome.	Use case: Support Assistant Bot Prompt instruction: “You are a helpful assistant supporting the IT department with employees’ requests, such as support tickets, licenses, or hardware inquiries. Follow the instructions below according to the type of request. If the user asks about … If the request is vague or incomplete, … If the request is about licenses or hardware, first … If the user wants to know about their support tickets, …”

In part two of this series, you will be able to find a shorter example of a Tree-Of-Thought prompt with Function Calling in action retrieved from the Support Assistant Starter App.

From basics to more complex problems: Starting your journey of prompting like a pro

As use cases get more complex and lack clear context, models are more likely to produce fabricated or irrelevant information, known as “hallucinations.” In cases where less background is needed, this issue is far less common. Mitigating these hallucinations is one of the great challenges of prompt engineering, so let’s dive into a journey to explore how this can be accomplished.

To begin with, let us consider a brief example of having a chatbot that answers questions. You would start writing:

You are a helpful assistant who helps users with their questions and requests.

Now, imagine you are trying to create a bot for a company and want to give some context. You might write something like:

You are a helpful assistant who provides information about Mendix. 
Please collect any publicly available information about the company for the user’s question.

Although this prompt should help the user get publicly available information about a specific company, it might start hallucinating when the request becomes more technical or the answer is not, or cannot be easily, accessible.

So, you start giving instructions for better user experience, such as:

You are a helpful assistant who provides information about Mendix. 
If the user has a technical question, include the Mendix documentation link. 
If the user is struggling with a bug, check Mendix Forum or Documentation for a solution. 
Please provide the source of the information in your response. 
Lastly, if you are not sure about the response, do not try to create one but rather inform the user that you do not know the answer.

From here, you can start enhancing the system prompt with other types, such as a Few-Shot Prompt or an Instruction Prompt:

You are a helpful assistant who provides information about Mendix.

1. For technical questions, include a link to the Mendix documentation. 
Example: If the user asks about connecting to OpenAI, you can answer: 
“You can connect your Mendix App with OpenAI by using the OpenAI Connector. 
More information can be found here: https://docs.mendix.com/appstore/modules/genai/openai/.”

2. For bug-related issues, refer to the Mendix Forum or Documentation. 
Example: If the user says that they created their first app and have an encryption issue, you can answer 
“An encryption key might be missing. 
More information can be found here: https://community.mendix.com/link/space/app-development/questions/99504..”

3. If unsure about a response, inform the user rather than guessing. 
Example: “I do not know the answer. Please provide more information or check the Mendix Documentation (https://docs.mendix.com/) for an answer.”

Certainly, searching public information for each request might not be the most feasible solution, as it allows the model more room for interpretation, increasing the likelihood of hallucinations. Additionally, it limits its knowledge to a certain period and does not give up-to-date data. To mitigate this, integrating the model with a company’s knowledge base can provide a controlled scope for seeking answers, thereby reducing the chances of false responses, and providing the latest or live information. We will explore this further in the next post.

If you are working on your own GenAI use case and need help or would like to provide feedback, we would love to hear from you. Get in touch with your Customer Success Manager or send us a message in the #genai-connectors channel in the Mendix Community Slack. Sign up here!

Additional resources to check out: Prompt library

Developer Blog