LLM: how to configure each type of model ?

Once one or more collections have been created in the External Content menu, it is possible to configure an LLM model.

Summary of my LLM settings

The “Summary of my LLM settings” modal allows you to configure the parameters needed for the operation of the LLM that will be used.

In the modal above, you can enter the following parameters depending on the chosen LLM type by clicking the Edit button:

1. LLM model type :

   • Azure OpenAI Dydu

   • OpenAI

   • Azure OpenAI

   • MistralAI

   • GoogleAI Gemini

   • VertexAI Gemini

   • Scaleway

2. API authentication key : Enter the API authentication key associated with the LLM model you will use.

3. LLM and embedding model :

   1. LLM model: An LLM is an artificial intelligence model trained on large amounts of text to understand and generate natural language. It can translate, summarize, answer questions, etc., using context and word meaning.

   2. Embedding model: An embedding model transforms words or phrases into numeric vectors so that the AI can compare their meanings. Similar texts will have close vectors, which helps process and retrieve information more intelligently.

4. Additional options : Depending on the LLM model you use, there may be additional options to fill in.

Once the parameters are entered, simply click Apply and the model is ready to use.

Types of LLM models

From the dropdown list in “Summary of my LLM settings”, you can choose the one you want to use :

Azure OpenAI Dydu

The Azure OpenAI Dydu model is used by default, without the need to adjust any settings.

OpenAI

• Here is the link to the LLM and embedding models you can use.

• API version: v1 is the default version used by OpenAI for the endpoints, so you can leave it as v1 or use another version.

Azure OpenAI

• If you want to use an Azure configuration, you will need to create two deployments:

  • The first for the LLM model to use, for example: gpt-4o-mini

  • The second for embedding, for example: text-embedding-3-large

• The example above is for illustration purposes only. However, the different LLM models and indexing methods are constantly evolving. Here is the link to the LLM and embedding models.

• You need to log in to your Azure portal: https://portal.azure.com to fill in the required fields:

  • In your Azure OpenAI resource, you will find:

    • API authentication key: Go to: Keys and endpoints. You will see two keys, Key1 and Key2 , copy one of them.

    • Azure OpenAI endpoint: In the same section, you will also see the endpoint URL, such as : https://your-resource-name.openai.azure.com/

    • Deployment names: This is the name you gave to the model deployment, for example: gpt-4, chat, embedding-model, etc. Go to the “Deployments” tab to find it.

    • API version: You can see this in the official Azure OpenAI documentation or in the portal.

MistralAI

Example of LLM and embedding model:

• LLM model : ministral-8b-latest

• Embedding model : mistral-embed

The example above is for illustration purposes only. However, LLM models and indexing methods are constantly evolving. Here is the link to the LLM models and the link to the embedding models.

GoogleAI Gemini

Example of LLM and embedding model:

• LLM model: gemini-1.5-flash-latest

• Embedding model: models/text-embedding-004

The example above is for illustration purposes only. However, LLM models and indexing methods are constantly evolving. Here is the link to the LLM and embedding models.

VertexAI Gemini

To configure the VertexAI Gemini settings, please refer to the following page: VertexAI Gemini

Scaleway

Here is the link to the LLM and embedding models.

Custom LLM

If you want to use your own custom LLM, which does not match any of the types mentioned above, you can manually configure your instance.

This configuration requires a few prerequisites:

• Have a URL exposing the routes specified in the table below (e.g.: https://my-api.test.fr/openai, allowing access to the /openai/chat/completion or /openai/embeddings endpoints)

• Provide an API key via the Authorization header (format: Bearer)

• Specify the LLM and embedding models you want to use (this information is required on our side)