Skip to content

Commit 09366e6

Browse files
committed
fix skypilot yaml
1 parent 922778e commit 09366e6

File tree

1 file changed

+12
-1
lines changed

1 file changed

+12
-1
lines changed

skypilot-tmp.yaml

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ envs:
1212
MODEL_NAME: Qwen/Qwen2.5-VL-32B-Instruct-AWQ
1313
HF_TOKEN: TOKEN_HERE
1414
VLLM_USE_V1: 1
15+
PYTHONUTF8: 1
1516

1617
resources:
1718
cloud: gcp
@@ -49,4 +50,14 @@ run: |
4950
--model $MODEL_NAME \
5051
--host 0.0.0.0 \
5152
--max-model-len 64000 \
52-
--limit-mm-per-prompt "image=15"
53+
--limit-mm-per-prompt '{"image":15}'
54+
55+
# curl http://SERVER_IP:8000/v1/chat/completions \
56+
# -H "Content-Type: application/json" \
57+
# -d '{
58+
# "model": "Qwen/Qwen2.5-VL-32B-Instruct-AWQ",
59+
# "messages": [
60+
# {"role": "system", "content": "You are a helpful assistant."},
61+
# {"role": "user", "content": "What is 2 + 2?"}
62+
# ]
63+
# }'

0 commit comments

Comments
 (0)