julian/amd-strix-halo-toolboxes
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

updated with dual-server benchmarks

Browse Source
This commit is contained in:
Donato Capitella
2026-01-12 13:19:23 +00:00
parent 7268e95b0f
commit 6d70dfc73b
33 changed files with 1841 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm6_4_4__hblt0__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 18.41 ± 0.00 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 4.12 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm6_4_4__hblt0__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 101.82 ± 0.34 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.71 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm6_4_4__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 17.93 ± 0.00 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 4.13 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm6_4_4__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 95.55 ± 0.26 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.78 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7-nightlies__hblt0__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 18.59 ± 0.00 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 3.63 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7-nightlies__hblt0__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 103.11 ± 0.08 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 9.11 ± 0.03 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7-nightlies__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 18.03 ± 0.00 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 3.63 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7-nightlies__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 87.98 ± 0.29 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 9.10 ± 0.02 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_1_1__hblt0__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 18.20 ± 0.00 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 3.60 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_1_1__hblt0__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 102.58 ± 0.04 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.74 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_1_1__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 17.88 ± 0.00 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 3.58 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__rocm7_1_1__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 93.44 ± 0.29 |
| glm4moe 355B.A32B Q4_K - Medium | 189.69 GiB | 356.79 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 8.71 ± 0.01 |
build: 2656c0d26 (7693)
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__vulkan_amdvlk__longctx32768__rpc.log
+19
View File
@@ -0,0 +1,19 @@
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Radeon 8060S Graphics (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-rpc/ggml-rpc.cpp:724: Remote RPC server crashed or returned malformed response
/lib64/libggml-base.so.0(+0x35a5) [0x7f92f39eb5a5]
/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7f92f39eb96b]
/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7f92f39ebaef]
/lib64/libggml-rpc.so.0(+0x5b4a) [0x7f92f7010b4a]
/lib64/libggml-base.so.0(+0x171b2) [0x7f92f39ff1b2]
/lib64/libggml-base.so.0(+0x1749f) [0x7f92f39ff49f]
/lib64/libggml-base.so.0(ggml_backend_alloc_ctx_tensors_from_buft+0x19) [0x7f92f3a00509]
/lib64/libllama.so.0(_ZN11llama_model12load_tensorsER18llama_model_loader+0x3c61) [0x7f92f72603c1]
/lib64/libllama.so.0(+0x25568) [0x7f92f71b6568]
/lib64/libllama.so.0(llama_model_load_from_file+0xac) [0x7f92f71b73cc]
/usr/sbin/llama-bench() [0x4077b5]
/lib64/libc.so.6(+0x35b5) [0x7f92f33815b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7f92f3381668]
/usr/sbin/llama-bench() [0x409cf5]
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__vulkan_amdvlk__rpc.log
+19
View File
@@ -0,0 +1,19 @@
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Radeon 8060S Graphics (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-rpc/ggml-rpc.cpp:724: Remote RPC server crashed or returned malformed response
/lib64/libggml-base.so.0(+0x35a5) [0x7f4efadba5a5]
/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7f4efadba96b]
/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7f4efadbaaef]
/lib64/libggml-rpc.so.0(+0x5b4a) [0x7f4efe3dfb4a]
/lib64/libggml-base.so.0(+0x171b2) [0x7f4efadce1b2]
/lib64/libggml-base.so.0(+0x1749f) [0x7f4efadce49f]
/lib64/libggml-base.so.0(ggml_backend_alloc_ctx_tensors_from_buft+0x19) [0x7f4efadcf509]
/lib64/libllama.so.0(_ZN11llama_model12load_tensorsER18llama_model_loader+0x3c61) [0x7f4efe62f3c1]
/lib64/libllama.so.0(+0x25568) [0x7f4efe585568]
/lib64/libllama.so.0(llama_model_load_from_file+0xac) [0x7f4efe5863cc]
/usr/sbin/llama-bench() [0x4077b5]
/lib64/libc.so.6(+0x35b5) [0x7f4efa7505b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7f4efa750668]
/usr/sbin/llama-bench() [0x409cf5]
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__vulkan_radv__longctx32768__rpc.log
+1
View File
@@ -0,0 +1 @@
Error: unable to find user kyuz0: no matching entries in passwd file
benchmark/results-rpc/GLM-4.6-UD-Q4_K_XL-00001-of-00005__vulkan_radv__rpc.log
+1
View File
@@ -0,0 +1 @@
Error: unable to find user kyuz0: no matching entries in passwd file
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm6_4_4__hblt0__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 59.80 ± 0.00 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 6.45 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm6_4_4__hblt0__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 172.78 ± 2.43 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.17 ± 0.05 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm6_4_4__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 59.95 ± 0.00 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 6.45 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm6_4_4__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 173.98 ± 1.76 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.17 ± 0.04 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7-nightlies__hblt0__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 60.12 ± 0.00 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 6.04 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7-nightlies__hblt0__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 157.51 ± 1.13 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.24 ± 0.10 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7-nightlies__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 60.47 ± 0.00 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 6.04 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7-nightlies__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 162.36 ± 1.16 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 18.23 ± 0.08 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_1_1__hblt0__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 62.07 ± 0.00 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 6.40 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_1_1__hblt0__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 160.68 ± 0.44 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.97 ± 0.02 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_1_1__longctx32768__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | pp2048 @ d32768 | 61.38 ± 0.00 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 2048 | 1 | 0 | tg32 @ d32768 | 6.40 ± 0.00 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__rocm7_1_1__rpc.log
+8
View File
@@ -0,0 +1,8 @@
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | pp512 | 159.35 ± 0.53 |
| minimax-m2 230B.A10B Q6_K | 180.94 GiB | 228.69 B | ROCm,RPC | 99 | 1 | 0 | tg128 | 17.98 ± 0.03 |
build: 2656c0d26 (7693)
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__vulkan_amdvlk__longctx32768__rpc.log
+3
View File
@@ -0,0 +1,3 @@
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Radeon 8060S Graphics (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
Failed to connect to 10.0.0.1:50052
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__vulkan_amdvlk__rpc.log
+19
View File
@@ -0,0 +1,19 @@
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Radeon 8060S Graphics (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-rpc/ggml-rpc.cpp:724: Remote RPC server crashed or returned malformed response
/lib64/libggml-base.so.0(+0x35a5) [0x7f7c046f25a5]
/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7f7c046f296b]
/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7f7c046f2aef]
/lib64/libggml-rpc.so.0(+0x5b4a) [0x7f7c07d17b4a]
/lib64/libggml-base.so.0(+0x171b2) [0x7f7c047061b2]
/lib64/libggml-base.so.0(+0x1749f) [0x7f7c0470649f]
/lib64/libggml-base.so.0(ggml_backend_alloc_ctx_tensors_from_buft+0x19) [0x7f7c04707509]
/lib64/libllama.so.0(_ZN11llama_model12load_tensorsER18llama_model_loader+0x3c61) [0x7f7c07f673c1]
/lib64/libllama.so.0(+0x25568) [0x7f7c07ebd568]
/lib64/libllama.so.0(llama_model_load_from_file+0xac) [0x7f7c07ebe3cc]
/usr/sbin/llama-bench() [0x4077b5]
/lib64/libc.so.6(+0x35b5) [0x7f7c040885b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7f7c04088668]
/usr/sbin/llama-bench() [0x409cf5]
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__vulkan_radv__longctx32768__rpc.log
+19
View File
@@ -0,0 +1,19 @@
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Radeon 8060S Graphics (RADV GFX1151) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
Failed to connect to 10.0.0.1:50052
radv/amdgpu: Failed to allocate a buffer:
radv/amdgpu: size : 990904320 bytes
radv/amdgpu: alignment : 262144 bytes
radv/amdgpu: domains : 4
radv/amdgpu: Failed to allocate a buffer:
radv/amdgpu: size : 990904320 bytes
radv/amdgpu: alignment : 262144 bytes
radv/amdgpu: domains : 4
radv/amdgpu: Failed to allocate a buffer:
radv/amdgpu: size : 990904320 bytes
radv/amdgpu: alignment : 262144 bytes
radv/amdgpu: domains : 4
radv/amdgpu: Failed to allocate a buffer:
radv/amdgpu: size : 990904320 bytes
radv/amdgpu: alignment : 262144 bytes
radv/amdgpu: domains : 4
benchmark/results-rpc/MiniMax-M2-UD-Q6_K_XL-00001-of-00004__vulkan_radv__rpc.log
+19
View File
@@ -0,0 +1,19 @@
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Radeon 8060S Graphics (RADV GFX1151) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | mmap | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | ---: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-rpc/ggml-rpc.cpp:724: Remote RPC server crashed or returned malformed response
/lib64/libggml-base.so.0(+0x35a5) [0x7fe6965fe5a5]
/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7fe6965fe96b]
/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7fe6965feaef]
/lib64/libggml-rpc.so.0(+0x5b4a) [0x7fe699c23b4a]
/lib64/libggml-base.so.0(+0x171b2) [0x7fe6966121b2]
/lib64/libggml-base.so.0(+0x1749f) [0x7fe69661249f]
/lib64/libggml-base.so.0(ggml_backend_alloc_ctx_tensors_from_buft+0x19) [0x7fe696613509]
/lib64/libllama.so.0(_ZN11llama_model12load_tensorsER18llama_model_loader+0x3c61) [0x7fe699e733c1]
/lib64/libllama.so.0(+0x25568) [0x7fe699dc9568]
/lib64/libllama.so.0(llama_model_load_from_file+0xac) [0x7fe699dca3cc]
/usr/sbin/llama-bench() [0x4077b5]
/lib64/libc.so.6(+0x35b5) [0x7fe695f945b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7fe695f94668]
/usr/sbin/llama-bench() [0x409cf5]
docs/results.json
+1549 -1
View File
File diff suppressed because it is too large Load Diff