Ollama Dev Service

When your application uses Spring AI to consume large language models from Ollama, you can use the Ollama Dev Service to start automatically an Ollama service. This enables you to develop and test your LLM-powered application without having to manually provide a model inference service.

Dependencies

First, you need to add the Ollama Dev Service dependency to your project. You can also include the Spring Boot DevTools dependency to enable live reload of your application during development.

dependencies {
  developmentOnly "org.springframework.boot:spring-boot-devtools"
  testAndDevelopmentOnly "io.arconia:arconia-dev-services-ollama"
}

When you use the Spring Boot DevTools in your project, Arconia will keep the Dev Services running while you make changes to your code instead of restarting them with the application. This allows you to see the changes in real-time without having to restart the Dev Services.

Running the Application

By default, the Ollama Dev Service is disabled since it might be resource-intensive, and running native Ollama might be more suitable for development. You can enable it globally or selectively for specific tests.

arconia:
  dev:
    services:
      ollama:
        enabled: true

You can enable/disable the Dev Service for a specific test class by using the @TestProperty annotation or equivalent Spring testing utilities.

You can enable/disable the Dev Service for a specific application mode (development, test, production), relying on one of the profiles which are automatically configured by Arconia (see Profiles).

Unlike the lower-level Testcontainers support in Spring Boot, Arconia doesn’t require special tasks to run your application when using Dev Services (./gradlew bootTestRun or ./mvnw spring-boot:test-run). You can simply run your application using the usual tasks provided by the Spring Boot plugins for Gradle or Maven.

Gradle: ./gradlew bootRun
Maven: ./mvnw spring-boot:run

The Dev Service will automatically start when you run your application. By default, the Dev Service will be reused across multiple applications, useful for reducing resource consumption, speeding up the startup time, and sharing large language models across multiple applications.

Your integration tests will automatically use the Dev Services without any additional configuration.

Customizing the Dev Service

You can customize the Dev Service via configuration properties, such as changing the image name or not reusing the service across multiple applications.

arconia:
  dev:
    services:
      ollama:
        image-name: ollama/ollama
        reusable: true