Implementation

Current Implementation

Overview

The Modelfile parser is implemented in src/core/modelfile.rs using the nom parser combinator library. It parses a text-based configuration format for defining model configurations, parameters, templates, and system prompts.

Grammar

The Modelfile grammar follows this structure:

command -> Instruction arguments*

Instruction -> "FROM" | "PARAMETER" | "TEMPLATE" | "SYSTEM" | "ADAPTER" | "LICENSE" | "MESSAGE" | "#"

arguments -> WORD | quoted_string | multiline_string

quoted_string -> "<str>"

multiline_string -> """<str>"""

Supported Instructions

FROM (Required)

The FROM instruction specifies the base model to use. This is the only required instruction in a Modelfile.

Syntax:

FROM <model_name>

Examples:

FROM llama3.2

FROM llama3.2:latest

FROM /path/to/model.gguf

FROM mlx-community/dolphin3.0-llama3.2-1B-4Bit

Validation:

  • Only one FROM instruction is allowed per Modelfile
  • A Modelfile without a FROM instruction will fail validation

PARAMETER

The PARAMETER instruction sets model inference parameters. Multiple parameters can be specified.

Syntax:

PARAMETER <parameter_name> <value>

Supported Parameters:

Integer Parameters:

  • num_ctx - Context window size
  • repeat_last_n - Number of tokens to consider for repeat penalty
  • seed - Random seed for generation
  • num_predict - Maximum number of tokens to predict
  • top_k - Top-k sampling parameter

Float Parameters:

  • temperature - Sampling temperature (0.0 to 1.0+)
  • repeat_penalty - Penalty for repeating tokens
  • top_p - Nucleus sampling parameter
  • min_p - Minimum probability threshold

String Parameters:

  • stop - Stop sequence (can be specified multiple times)

Examples:

PARAMETER num_ctx 4096

PARAMETER temperature 0.7

PARAMETER stop "<|eot_id|>"

PARAMETER stop "</s>"

PARAMETER top_p 0.9

PARAMETER seed 42

Validation:

  • Parameter names are case-insensitive
  • Integer parameters must parse as valid i32 values
  • Float parameters must parse as valid f32 values
  • Invalid parameter types will result in an error

TEMPLATE

The TEMPLATE instruction defines the prompt template for the model. Only one template is allowed.

Syntax:

TEMPLATE <template_string>

Examples:

TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""

Validation:

  • Only one TEMPLATE instruction is allowed per Modelfile
  • Template can be specified as a single-line string, quoted string, or multiline string

SYSTEM

The SYSTEM instruction sets the system prompt for the model. Only one system prompt is allowed.

Syntax:

SYSTEM <system_prompt>

Examples:

SYSTEM "You are a helpful assistant."

SYSTEM """

You are a bot

You also not a bot

"""

Validation:

  • Only one SYSTEM instruction is allowed per Modelfile
  • System prompt can be specified as a single-line string, quoted string, or multiline string

ADAPTER

The ADAPTER instruction specifies a path to a LoRA adapter file. Only one adapter is allowed.

Syntax:

ADAPTER <adapter_path>

Examples:

ADAPTER /path/to/adapter.bin

ADAPTER ./adapters/my_adapter.safetensors

Validation:

  • Only one ADAPTER instruction is allowed per Modelfile

LICENSE

The LICENSE instruction specifies the license text for the model. Only one license is allowed.

Syntax:

LICENSE <license_text>

Examples:

LICENSE "MIT"

LICENSE """

Apache License

Version 2.0, January 2004

...

"""

Validation:

  • Only one LICENSE instruction is allowed per Modelfile
  • License text can be specified as a single-line string, quoted string, or multiline string

MESSAGE

The MESSAGE instruction defines example messages for few-shot learning or conversation examples. Multiple messages can be specified.

Syntax:

MESSAGE <role> <message_content>

Supported Roles:

  • system - System message
  • user - User message
  • assistant - Assistant message

Examples:

MESSAGE user """Is Toronto in Canada?

or Is cologne in france"""

MESSAGE assistant yes

MESSAGE user Is Sacramento in Canada?

MESSAGE assistant no

Validation:

  • Role must be one of: system, user, or assistant (case-insensitive)
  • Invalid roles will result in an error

Comments

Comments are supported using the # character. Comments are preserved in the parsed Modelfile.

Syntax:

# <comment_text>

Examples:

# Modelfile generated by "ollama show"

# To build a new Modelfile based on this one, replace the FROM line with:

FROM llama3.2

# This is a comment

String Formats

The parser supports three string formats for instruction arguments:

  1. Single-line strings: Plain text until end of line

    FROM llama3.2

  2. Quoted strings: Text enclosed in double quotes "

    SYSTEM "You are a helpful assistant."

  3. Multiline strings: Text enclosed in triple quotes """

    SYSTEM """

You are a bot

You also not a bot

"""
ProposalReference