julian/amd-strix-halo-toolboxes
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

updated benchmarks

Browse Source
This commit is contained in:
Donato Capitella
2025-11-17 23:02:56 +00:00
parent ad32126872
commit 1d6d48fae1
192 changed files with 13571 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm6_4_4-rocwmma__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 64.83 ± 0.23 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.69 ± 0.01 |
build: caca0d55c (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm6_4_4-rocwmma__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 50.19 ± 0.10 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.73 ± 0.01 |
build: caca0d55c (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm6_4_4__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 66.04 ± 0.17 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.73 ± 0.01 |
build: 86f1f4411 (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm6_4_4__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 50.78 ± 0.06 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.72 ± 0.00 |
build: 86f1f4411 (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_1-rocwmma__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 62.70 ± 0.13 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.71 ± 0.01 |
build: f1840a25d (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_1-rocwmma__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 45.01 ± 0.11 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.73 ± 0.01 |
build: f1840a25d (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_1__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 65.83 ± 0.13 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.72 ± 0.02 |
build: 677be4d78 (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_1__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 42.96 ± 0.13 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.73 ± 0.01 |
build: 677be4d78 (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_alpha-rocwmma-improved__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 61.63 ± 0.11 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.92 ± 0.01 |
build: 12bb5c37 (7074)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_alpha-rocwmma-improved__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 36.76 ± 0.06 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.93 ± 0.01 |
build: 12bb5c37 (7074)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_alpha-rocwmma__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 66.33 ± 0.03 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 9.04 ± 0.01 |
build: 4db63cdde (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_alpha-rocwmma__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 30.17 ± 0.09 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 9.05 ± 0.01 |
build: 4db63cdde (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_alpha__hblt0__rpc.log
+21
View File
@@ -0,0 +1,21 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-rpc/ggml-rpc.cpp:858: Remote RPC server crashed or returned malformed response
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7fd18621c565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7fd18621c92b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7fd18621caaf]
/usr/local/lib64/libggml-rpc.so.0(+0xa195) [0x7fd1862ca195]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x7f3) [0x7fd186236de3]
/usr/local/lib64/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa0) [0x7fd189269650]
/usr/local/lib64/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0xe2) [0x7fd18926b2e2]
/usr/local/lib64/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x3bf) [0x7fd1892701bf]
/usr/local/lib64/libllama.so.0(llama_decode+0xe) [0x7fd18927100e]
/usr/local/bin/llama-bench() [0x40a3db]
/usr/local/bin/llama-bench() [0x407edc]
/lib64/libc.so.6(+0x35b5) [0x7fd185bb25b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7fd185bb2668]
/usr/local/bin/llama-bench() [0x409255]
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_alpha__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 31.42 ± 0.09 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 9.08 ± 0.01 |
build: 4fc43d43d (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_rc-rocwmma__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 63.23 ± 0.18 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.70 ± 0.01 |
build: b447a9a4b (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_rc-rocwmma__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 30.12 ± 0.09 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.72 ± 0.01 |
build: b447a9a4b (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_rc__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 63.17 ± 0.13 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.72 ± 0.01 |
build: fa5c85a8b (7085)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_rc__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 36.22 ± 0.08 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.71 ± 0.01 |
build: fa5c85a8b (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm6_4_4-rocwmma__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 172.03 ± 0.73 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.02 ± 0.02 |
build: caca0d55c (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm6_4_4-rocwmma__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 174.52 ± 1.29 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.94 ± 0.07 |
build: caca0d55c (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm6_4_4__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 171.93 ± 1.16 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.06 ± 0.03 |
build: 86f1f4411 (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm6_4_4__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 174.45 ± 1.01 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.02 ± 0.03 |
build: 86f1f4411 (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_1-rocwmma__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 170.89 ± 0.37 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.89 ± 0.08 |
build: f1840a25d (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_1-rocwmma__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 173.53 ± 1.57 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.92 ± 0.05 |
build: f1840a25d (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_1__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 172.01 ± 0.91 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.95 ± 0.04 |
build: 677be4d78 (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_1__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 173.90 ± 0.67 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.95 ± 0.03 |
build: 677be4d78 (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_alpha-rocwmma-improved__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 142.82 ± 0.78 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.85 ± 0.08 |
build: 12bb5c37 (7074)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_alpha-rocwmma-improved__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 144.47 ± 1.03 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.86 ± 0.03 |
build: 12bb5c37 (7074)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_alpha-rocwmma__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 143.05 ± 1.08 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.00 ± 0.05 |
build: 4db63cdde (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_alpha-rocwmma__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 145.60 ± 1.03 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.94 ± 0.02 |
build: 4db63cdde (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_alpha__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 144.59 ± 0.50 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.01 ± 0.03 |
build: 4fc43d43d (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_alpha__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 146.21 ± 2.31 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.05 ± 0.10 |
build: 4fc43d43d (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_rc-rocwmma__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 170.42 ± 0.65 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.88 ± 0.06 |
build: b447a9a4b (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_rc-rocwmma__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 174.42 ± 0.52 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.87 ± 0.12 |
build: b447a9a4b (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_rc__hblt0__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 171.34 ± 0.97 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.97 ± 0.01 |
build: fa5c85a8b (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_rc__rpc.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 173.98 ± 1.06 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.94 ± 0.03 |
build: fa5c85a8b (7085)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__vulkan_amdvlk__rpc.log
+18
View File
@@ -0,0 +1,18 @@
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Radeon 8060S Graphics (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-rpc/ggml-rpc.cpp:690: Remote RPC server crashed or returned malformed response
/lib64/libggml-base.so.0(+0x3565) [0x7f5d2cbe9565]
/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7f5d2cbe992b]
/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7f5d2cbe9aaf]
/lib64/libggml-rpc.so.0(+0x452a) [0x7f5d2fb8252a]
/lib64/libggml-base.so.0(+0x16232) [0x7f5d2cbfc232]
/lib64/libggml-base.so.0(ggml_backend_alloc_ctx_tensors_from_buft+0xff) [0x7f5d2cbfdf1f]
/lib64/libllama.so.0(_ZN11llama_model12load_tensorsER18llama_model_loader+0x3a26) [0x7f5d2fdaad06]
/lib64/libllama.so.0(+0x1cf16) [0x7f5d2fd11f16]
/lib64/libllama.so.0(llama_model_load_from_file+0xac) [0x7f5d2fd12d7c]
/usr/sbin/llama-bench() [0x406d85]
/lib64/libc.so.6(+0x35b5) [0x7f5d2c57f5b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7f5d2c57f668]
/usr/sbin/llama-bench() [0x409255]