Uncovering GenAI - Chapter 3: Building a Local LLM-Powered Application with Prompt Engineering

            sequenceDiagram
                actor User
                participant App as Console
Application
(.NET)
                participant LLM as Ollama
LLM
                loop
                    User->>App: Inputs a prompt/question
                    App->>App: Add purpose context
to conversation history
                    App->>App: Add User's prompt
to conversation history
                    App->>LLM: Send conversation history
and prompt to local LLM
                    LLM-->>App: Provide a response back
                    App->>App: Add AI response
to conversation history
                    App-->>User: Display AI response
                    App->>User: Ask user for the
next prompt/question
                end
        

With the project skeleton in place, we need to wire up the application. For this, we will have to do a couple of things:

• Create a ChatApplication class and use the constructor to set up the application host and configurations. The configuration setup will allow reading settings from appsettings.json in the next step.
• Since we have hooked up the appsettings.json file in the previous step, we will read our Ollama Endpoint and Chat Model from the configuration.
• Now we will register a service with dependency injection that lets our application use a local Ollama chat completion model (our downloaded LLM model in the prerequisites step) for generating responses. This is done by leveraging the out-of-the-box AddOllamaChatCompletion method that the Semantic Kernel packages add to the IServiceCollection.
• We will now register the Kernel class as a transient service in the dependency injection container. This will allow dependency injection to resolve the Kernel class on demand.
• Finally, we will build the host and expose a method to get the service from the host's service provider.

            
public class ChatApplication
{
    private readonly IHost _host;

    public ChatApplication()
    {
        var hostBuilder = Host.CreateDefaultBuilder();
        hostBuilder.ConfigureAppConfiguration(cfg => { cfg.AddJsonFile("appsettings.json", false, true); });

        hostBuilder.ConfigureServices((context, services) =>
        {
            var chatEndpoint = new Uri(context.Configuration["Ollama:Endpoint"] ?? string.Empty);
            var model = context.Configuration["Ollama:ChatModel"] ?? string.Empty;
            services.AddOllamaChatCompletion(model, chatEndpoint);
            services.AddTransient(serviceProvider => new Kernel(serviceProvider));
        });

        _host = hostBuilder.Build();
    }

    public T GetService() where T : notnull
    {
        return _host.Services.GetRequiredService();
    }
}