OurBigBook.com $£ Sponsor 中国独裁统治 China Dictatorship 新疆改造中心、六四事件、法轮功、郝海东、709大抓捕、2015巴拿马文件邓家贵、低端人口、西藏骚乱

ollama-expect

Usage:

./ollama-expect <model> <prompt>

e.g.:

./ollama-expect llama3.2 'What is quantum field theory?'

This generates 100 tokens for the given prompt with the given model.

Benchmarks:

P14s: 4.8s, CPU only: ~21 tokens / s. For comparison, using the Vulkan backend of llama.cpp gave ~23 tokens/s
P51: 9.6s, uses Nvidia GPU: ~10 tokens / s

ollama-expect

#!/usr/bin/expect -f
set prompt ">>> "
log_user 0
spawn ollama run [lindex $argv 0]
expect $prompt
send "/set parameter temperature 0\r"
expect $prompt
send "/set parameter num_predict 100\r"
expect $prompt
send "[lindex $argv 1]\r"
expect -re "\n(.*?)$prompt"
puts -nonewline $expect_out(1,string)
send -- "/bye"

 Ancestors (17)

 Incoming links (3)