Introduction

Artificial Intelligence is becoming an essential part of modern software development. However, cloud-based AI APIs can involve subscription costs, API keys, internet dependency, and data privacy considerations. Running a model locally offers another way to experiment with AI directly on your own computer.

In this tutorial, we will set up and run Qwen2.5 0.5B locally using Ollama. We will cover installation, model download, interactive execution, API verification, and testing requests with cURL and PowerShell.

What You Will Learn

Install Ollama on Windows.
Download and run the Qwen2.5 0.5B model.
Verify the local Ollama API server.
Send prompts using the Generate and Chat APIs.
Understand the request-response flow and troubleshoot common issues.

1. What Is Ollama?

Ollama is a tool that makes it easier to download, manage, and run supported Large Language Models (LLMs) locally. It also exposes a local REST API that applications can use to send prompts and receive model responses.

Key Features

Run supported AI models locally.
Download and manage multiple models.
Access models through a local REST API.
Integrate local models into Python, JavaScript, React, Angular, Flask, and Node.js applications.
Perform inference without sending prompts to a cloud AI API.

Official website: https://ollama.com/

2. What Is Qwen2.5 0.5B?

Qwen2.5 0.5B is a lightweight language model from the Qwen family developed by Alibaba. It has approximately 0.5 billion parameters and is useful for local experimentation, prototyping, and learning how LLM-powered applications work.

Because it is a small model, it may respond faster and require fewer resources than larger models. However, it will generally be less capable than larger models on complex reasoning, coding, and advanced instruction-following tasks.

3. Prerequisites

A Windows computer with a supported 64-bit operating system.
An internet connection for the initial Ollama and model downloads.
Sufficient free disk space and available system memory.
Command Prompt or PowerShell.
A compatible GPU is optional; CPU execution is also possible, though performance may differ.

4. Step 1 — Install Ollama

Follow these steps to install Ollama on Windows:

Open a browser and visit https://ollama.com/.
Select the Download option and choose the Windows installer.
When the download completes, open the installer.
Follow the installation instructions and wait for setup to finish.
Open a new Command Prompt or PowerShell window.

Verify the Installation

Run the following command:

ollama –version

If Ollama is installed and available in your command-line PATH, the terminal will display the installed version.

5. Step 2 — Download and Run Qwen2.5 0.5B

Run this command in Command Prompt or PowerShell:

ollama run qwen2.5:0.5b

Ollama will:

Check whether the requested model is already available locally.
Download the model if it is not installed.
Load the model.
Start an interactive chat session.

Test the Model

When the interactive prompt appears, enter a question such as:

Hello! Who are you?

You can also try:

Explain Artificial Intelligence in simple words.

Exit the Interactive Session

Type the following command and press Enter:

/bye

List Downloaded Models

ollama list

This command displays the models currently available in your local Ollama library.

6. Step 3 — Verify the Local Ollama API Server

Ollama provides a local HTTP API. By default, the server listens on port 11434.

http://localhost:11434

Open the address in a browser. If the server is running, the page should display:

Ollama is running

The Ollama application or background service must be running for local API requests to work.

7. Step 4 — Test the Generate API Using cURL

The Generate API lets an application submit a prompt to a model and receive generated text. Open a new Command Prompt or PowerShell window and run:

curl http://localhost:11434/api/generate -d “{\”model\”:\”qwen2.5:0.5b\”,\”prompt\”:\”Explain Artificial Intelligence in simple words.\”,\”stream\”:false}”

Understanding the Request

Parameter	Description
model	The model to use for generating the response.
prompt	The input text or question sent to the model.
stream	When false, the response is returned as one complete JSON result.

Example Response

The API returns JSON. A simplified example is shown below; the actual wording will vary:

{
“model”: “qwen2.5:0.5b”,
“response”: “Artificial Intelligence enables machines to perform tasks that normally require human intelligence.”,
“done”: true
}

PowerShell Alternative

You can also send the request using PowerShell:

$body = @{
    model = “qwen2.5:0.5b”
    prompt = “What is Machine Learning?”
    stream = $false
} | ConvertTo-Json

Invoke-RestMethod `
    -Uri “http://localhost:11434/api/generate” `
    -Method Post `
    -ContentType “application/json” `
    -Body $body

8. Step 5 — Test the Chat API

Ollama also provides a Chat API designed for conversational messages. It accepts a messages array with roles such as system, user, and assistant.

curl http://localhost:11434/api/chat -d “{\”model\”:\”qwen2.5:0.5b\”,\”messages\”:[{\”role\”:\”user\”,\”content\”:\”What is Generative AI?\”}],\”stream\”:false}”

The Chat API is useful when building chatbots and applications that maintain a conversational message history.

9. Understanding the Local AI Architecture

The basic request-response flow can be represented as:

User Prompt → Application / Terminal → Ollama Local API
→ Qwen2.5 0.5B Model → Generated Response

The user enters a prompt.
The application sends an HTTP request to Ollama.
Ollama passes the request to the selected model.
Qwen2.5 0.5B processes the prompt and generates text.
Ollama returns the response to the application.

When using this local setup, model inference happens on your machine. Keep in mind that other application components may still use network services if you configure them to do so.

10. Common Issues and Troubleshooting

‘ollama’ Is Not Recognized

Possible cause: Ollama is not installed correctly, or its executable directory is not available in the system PATH.

Try the following:

Confirm that Ollama is installed.
Restart Command Prompt or PowerShell.
If needed, restart the computer and check the PATH configuration.

Connection Refused on Port 11434

Possible cause: the Ollama server is not running. Open the Ollama application, verify that the service is active, and retry http://localhost:11434/.

Model Download Is Taking Too Long

The first run downloads model files. Download time depends on your internet connection. Check connectivity and allow the download to finish.

Model Response Is Slow

Performance depends on CPU, available RAM, GPU support, model settings, and other running applications. Close unnecessary applications and ensure sufficient memory is available.

11. Advantages of Running AI Models Locally

Privacy: Prompts and inference can remain on your local machine.
No external AI API dependency: You do not need a cloud AI API for local inference.
Cost efficiency: You can experiment without per-request cloud API charges, although hardware and electricity still have costs.
Developer-friendly integration: The REST API makes it straightforward to connect local models to applications.
Offline inference: After installation and model download, inference can work without an internet connection.

12. Conclusion

In this tutorial, we installed Ollama, downloaded and ran Qwen2.5 0.5B, verified the local API server, and tested both Generate and Chat API requests. We also reviewed the local architecture and common troubleshooting steps.

Ollama provides a practical starting point for developers who want to learn about local LLMs and build AI-powered chatbots, automation tools, and application prototypes.

Start experimenting with local AI today!

Official Resources

Ollama Official Website: https://ollama.com/
Ollama Documentation: https://docs.ollama.com/
Qwen2.5 Model Library: https://ollama.com/library/qwen2.5
Ollama API Documentation: https://docs.ollama.com/api

Tutorial Details

Topic: Local AI Model Setup

Tool: Ollama

Model: Qwen2.5 0.5B

Platform: Windows

API: Ollama REST API

Testing tools: cURL and PowerShell

Difficulty: Beginner

Ollama with Qwen2.5 0.5B – Local AI Setup