Putting AI Prompt Engineering Into Practice—Part 2

Welcome to the second part of our blog services on AI Prompt Engineering where we examine the importance of prompt engineering when it is integrated into processes to automate tasks and achieve better outcomes.

Be sure to read our first blog post in this series, Understanding AI Prompt Engineering, where we lay the groundwork for how you can put this into practices in your organization’s business processes.

Using prompts and prompt engineering for automation

Our simple example in the previous blog post shows a human interacting directly with a chatbot in a “conversation” reviewing results and fine tuning right away. The approach is different when you take this concept and implement this type of service in a business process, but can include an iterative approach for the best results.

AI requires detailed instructions to mimic humans and create the most relevant and high-quality output. And now you understand that the objective of AI prompt engineering is to supply the AI with relevant context, clear instructions, and illustrative examples, enabling it to grasp the intent and generate meaningful responses.

There are other components that need to be considered in order to maximize results when integrating generative AI into your business operations. As you can imagine, these tools require quite a bit of horsepower, which converts to costs. In many cases, the selected LLM might be the key to the returned results. A common way to implement integration is using APIs.

When building integration with models into workflows using APIs, you will want to become familiar with some additional terms so that you can configure how your application will work with the LLM:

• Model ID or name. You will need to note which model you want to invoke when you are making an API call to a specific LMM. Sometimes a change in model can improve your results.
• System (full context). Where instructions are provided to the model or the actual “prompt” to the LLM. This is what is telling the underlying AI provider what to do. For example, create an email to respond to a customer inquiry.
• User. This is the input that we are providing for the prompt. For example, here is the customer inquiry I received to which I need a response.
• Maximum tokens. More tokens can allow the AI provider to tackle more complex tasks and provide more nuanced responses. It is also how you are charged by an AI provider. The more tokens, the higher the cost. For example, 1,000 tokens equate to about 750 words. The maximum tokens help to set a limit so you are not charged for excessively wordy responses. In most cases, 500 to 1,000 max tokens are appropriate depending on the use case.
• Temperature. This provides information on how “creative” the response can be—the higher the temperature, the more creative. A lower temperature indicates more consistent responses. Your selection will depend on your use case. Consistency is likely important and you want all your users to have a similar experience, so a lower temperature is indicated.

As with any API, you will need to have your authentication method and any appropriate access keys for that API.

Framing the task for optimal results

As discussed, the most important part of prompt engineering is providing all the right stuff in order to get optimal results. For example, when working with an API of a LMM model, you need to be sure to “tell” the model the specifics of its job and what role it is going to play in your operations. This provides the model with specifics about its job and role for this task. For example:

You are an email responder.

You also want to provide primary instructions or guidance about what is going to happen and the goals of the interaction. This may be in the form of primary and secondary goals or instructions.

I will provide you with an email that I receive and you will respond to that email. My name is James, I work at AIPrompt Solutions, and I am in the Customer Support group.

Steps: 
1. Comprehend what is wanted in the email received - reference the subject line and body 
2. Write a response to the email using the following format: Full email where the subject is identified with “Subject:” and body is identified with “Body:” 

You may even provide an output format and example output. These options depend on the API and model you elect to integrate using the API.

Output Format: When providing an output, do not use "Subject:" or "Body:", just provide the relevant information for each of those sections. 

Example Output: Hello [name], Thank you for your interest in our new AI  product. It features a simple interface, multiple generative AI models, advanced analytics, and customizable reports. For more detailed information, please see our product documentation. Let me know if you have any other questions. Thank you. Sincerely, James AIPrompt Solutions, Customer Support Specialist
Specific Instructions: Each email body should be a maximum of 4 sentences.

Providing all the correct components for guidance will set you up for success.

Note: The example shown was derived from this video.

Example of using this technique in a Camunda process

Let’s assume you work for a travel agency and you want to provide an experience for your customers that mimics a conversation that they might have with a travel agent for upcoming travel.

The customer is presented with a dialog to enter the details about the desired travel details.

As you can see, you are providing a place for the customer to put information that might help determine what is the best travel method for them. For example, they might put “I am a vegetarian,” or “I do not like to travel by air,” or “I prefer to travel in the early morning.” This type of information allows the LMM to include these details into consideration when providing travel suggestions.

You are also providing a place to put the travel details, for example:

I need to travel from Berlin, Germany to Barcelona, Spain leaving on Saturday, December 7, 2024 and returning on Thursday, December 12, 2024. I will not need a vehicle when in Spain. I do need lodging in Barcelona for this trip.

In this case, the following sample Camunda process is provided for explanation. Here, the traveler is prompted to enter the information shown in the form shown above. This information is sent to the loop (shown in the white box below) for processing by the correct AI agents..

The Travel Agent component uses the AWS Bedrock service to determine first, if the traveler has provided enough information to process the request and then which other agents (flight, hotel, car) should be invoked to provide a full picture of the options to the traveler.  

As part of the integration to the Bedrock API, you must provide the proper access and secret keys as well as the model to be used and the “payload” which includes our sophisticated prompt telling the LMM our expectations as well as how to provide the response.

In the response of this API call, we capture if certain agents need to be invoked. Based on the outcome of this travel agent engine, a setting is captured and a value is set for future agent invocation.

Let’s dissect the prompt provided to Bedrock in this example. Remember those terms we mentioned in the Using prompts and prompt engineering for automation section? This is a place where you see some of them—specifically, the max_tokens shown below.

Next, we can see the system prompt providing the LLM with the scope and parameters of the request. You can see that we are providing information about this Travel Agent acting as a routing agent for the travel request to assemble trips for the traveler.

Guidance is then provided using the role of the user giving guidance on what to do if the query is incomplete or missing information.

Finally, each subsequent invoked agent has a section in the payload for the model.

Email agent

Flight agent

Hotel agent

Car agent

All of these agents work together or separately depending on the requirements for the traveler.

When the traveler is happy with the selections provided, the booking agent can take over to book the itinerary. Keep in mind, the prompts shown in this example were “engineered” so that the selected LMM and other parameters would provide the best results.

Want more information?

When using generative AI models in your organization, prompt engineering is key to getting the most out of this automation and experience.

If you want to take a look at these files in more detail, please access this GitHub repository.