{"id":683,"date":"2024-06-01T23:16:33","date_gmt":"2024-06-01T14:16:33","guid":{"rendered":"https:\/\/blog.gurees.net\/?p=683"},"modified":"2024-06-15T17:14:37","modified_gmt":"2024-06-15T08:14:37","slug":"rocky-linux%e3%81%a8radeon-rx-480%e3%81%a7rocm%e3%82%92%e4%bd%bf%e3%81%a3%e3%81%a6llm%e3%82%92%e5%8b%95%e3%81%8b%e3%81%97%e3%81%a6%e3%81%bf%e3%82%8b","status":"publish","type":"post","link":"https:\/\/blog.gurees.net\/?p=683","title":{"rendered":"Rocky Linux\u3068Radeon RX 480\u3067ROCm\u3092\u4f7f\u3063\u3066LLM\u3092\u52d5\u304b\u3057\u3066\u307f\u308b"},"content":{"rendered":"\n<p>\u53e4\u3081\u306e\u74b0\u5883\u3067\u3059\u304cRyzen 7 1700\u3068RX 480\u306e\u74b0\u5883\u304c\u3042\u3063\u305f\u306e\u3067ROCm\u3067\u6a5f\u68b0\u5b66\u7fd2\u304c\u3067\u304d\u306a\u3044\u304b\u3068\u601d\u3063\u305f\u306e\u3067\u5b9f\u9a13\u3057\u3066\u307f\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u74b0\u5883\u306f\u4ee5\u4e0b\u306e\u81ea\u4f5cPC\u306bRocky Linux9.4\u3092Minimum\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u307e\u3057\u305f\u3002<br>CPU\uff1aRyzen 7 1700<br>MB\uff1aTUF B450M-PLUS GAMING<br>RAM\uff1a16GB\u00d72\u679a 2666 MHz<br>SSD\uff1aIntel 670p 1TB<\/p>\n\n\n\n<!--more-->\n\n\n\n<p>Radeon\u306e\u30c9\u30e9\u30a4\u30d0\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3059\u308b\u30c4\u30fc\u30eb\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u307e\u3059\u3002\u73fe\u6642\u70b9(2024\/6\/1)\u306e\u6700\u65b0\u7248\u306fROCm6\u7cfb\u3067\u3059\u304c\u3001RX480\u3092\u52d5\u304b\u3059\u90fd\u5408\u30675\u7cfb\u306e\u6700\u65b0\u7248\u3060\u3063\u305f5.7.3\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># dnf install https:\/\/repo.radeon.com\/amdgpu-install\/5.7.3\/rhel\/9.2\/amdgpu-install-5.7.50703-1.el9.noarch.rpm -y<\/pre>\n\n\n\n<p>EPEL\u30ea\u30dd\u30b8\u30c8\u30ea\u3068CodeReady Linux Builder\u30ea\u30dd\u30b8\u30c8\u30ea\u304c\u5fc5\u8981\u306a\u306e\u3067\u6709\u52b9\u306b\u3057\u3066\u304b\u3089\u3001amdgpu-install\u3092\u5b9f\u884c\u3057\u307e\u3059\u30025\u7cfb\u306e\u5bfe\u5fdc\u30d0\u30fc\u30b8\u30e7\u30f3\u304c9.2\u307e\u3067\u307f\u305f\u3044\u306a\u306e\u3067\u3001amdgpudistro\u3092\u30bb\u30c3\u30c8\u3057\u3066\u304a\u304d\u307e\u3059\u3002<br>\uff08\u6700\u521d\u306f9.2\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u305f\u3093\u3067\u3059\u304c\u3001dnf update\u3057\u305f\u3089\u30d0\u30fc\u30b8\u30e7\u30f3\u304c\u4e0a\u304c\u3063\u3066\u3057\u307e\u3044\u307e\u3057\u305f\uff09<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># echo \"9.2\" > \/etc\/yum\/vars\/amdgpudistro\n# dnf install epel-release -y\n# crb enable\n# amdgpu-install --usecase=rocm<\/pre>\n\n\n\n<p>\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u306b2.4G\u7a0b\u5ea6\u3001\u30c7\u30a3\u30b9\u30af\u308217G\u7a0b\u5ea6\u8981\u6c42\u3055\u308c\u305f\u306e\u3067\u3001\u3057\u3070\u3089\u304f\u5f85\u3061\u307e\u3059\u3002<\/p>\n\n\n\n<p>dkms\u3067\u30e2\u30b8\u30e5\u30fc\u30eb\u3092\u78ba\u8a8d\u3059\u308b\u3068added\u3068\u306a\u3063\u3066\u304a\u308a\u3001rocminfo\u3067\u78ba\u8a8d\u3057\u3066\u3082ROCk module is loaded\u3068\u306a\u3063\u3066\u3044\u308b\u306e\u3067\u3001\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u306f\u3067\u304d\u305f\u3088\u3046\u3067\u3059\u3002<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># dkms status\namdgpu\/6.2.4-1697730.el9: added\n# rocminfo\nROCk module is loaded\n=====================\nHSA System Attributes\n=====================\nRuntime Version:         1.1\nSystem Timestamp Freq.:  1000.000000MHz\nSig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)\nMachine Model:           LARGE\nSystem Endianness:       LITTLE\nMwaitx:                  DISABLED\nDMAbuf Support:          YES\n\n==========\nHSA Agents\n==========\n*******\nAgent 1\n*******\n  Name:                    AMD Ryzen 7 1700 Eight-Core Processor\n  Uuid:                    CPU-XX\n  Marketing Name:          AMD Ryzen 7 1700 Eight-Core Processor\n  Vendor Name:             CPU\n  Feature:                 None specified\n  Profile:                 FULL_PROFILE\n  Float Round Mode:        NEAR\n  Max Queue Number:        0(0x0)\n  Queue Min Size:          0(0x0)\n  Queue Max Size:          0(0x0)\n  Queue Type:              MULTI\n  Node:                    0\n  Device Type:             CPU\n  Cache Info:\n    L1:                      32768(0x8000) KB\n  Chip ID:                 0(0x0)\n  ASIC Revision:           0(0x0)\n  Cacheline Size:          64(0x40)\n  Max Clock Freq. (MHz):   3000\n  BDFID:                   0\n  Internal Node ID:        0\n  Compute Unit:            16\n  SIMDs per CU:            0\n  Shader Engines:          0\n  Shader Arrs. per Eng.:   0\n  WatchPts on Addr. Ranges:1\n  Features:                None\n  Pool Info:\n    Pool 1\n      Segment:                 GLOBAL; FLAGS: FINE GRAINED\n      Size:                    32508848(0x1f00bb0) KB\n      Allocatable:             TRUE\n      Alloc Granule:           4KB\n      Alloc Alignment:         4KB\n      Accessible by all:       TRUE\n    Pool 2\n      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED\n      Size:                    32508848(0x1f00bb0) KB\n      Allocatable:             TRUE\n      Alloc Granule:           4KB\n      Alloc Alignment:         4KB\n      Accessible by all:       TRUE\n    Pool 3\n      Segment:                 GLOBAL; FLAGS: COARSE GRAINED\n      Size:                    32508848(0x1f00bb0) KB\n      Allocatable:             TRUE\n      Alloc Granule:           4KB\n      Alloc Alignment:         4KB\n      Accessible by all:       TRUE\n  ISA Info:\n*******\nAgent 2\n*******\n  Name:                    gfx803\n  Uuid:                    GPU-XX\n  Marketing Name:          AMD Radeon (TM) RX 480 Graphics\n  Vendor Name:             AMD\n  Feature:                 KERNEL_DISPATCH\n  Profile:                 BASE_PROFILE\n  Float Round Mode:        NEAR\n  Max Queue Number:        128(0x80)\n  Queue Min Size:          64(0x40)\n  Queue Max Size:          131072(0x20000)\n  Queue Type:              MULTI\n  Node:                    1\n  Device Type:             GPU\n  Cache Info:\n    L1:                      16(0x10) KB\n  Chip ID:                 26591(0x67df)\n  ASIC Revision:           1(0x1)\n  Cacheline Size:          64(0x40)\n  Max Clock Freq. (MHz):   1266\n  BDFID:                   1792\n  Internal Node ID:        1\n  Compute Unit:            36\n  SIMDs per CU:            4\n  Shader Engines:          4\n  Shader Arrs. per Eng.:   1\n  WatchPts on Addr. Ranges:4\n  Features:                KERNEL_DISPATCH\n  Fast F16 Operation:      TRUE\n  Wavefront Size:          64(0x40)\n  Workgroup Max Size:      1024(0x400)\n  Workgroup Max Size per Dimension:\n    x                        1024(0x400)\n    y                        1024(0x400)\n    z                        1024(0x400)\n  Max Waves Per CU:        40(0x28)\n  Max Work-item Per CU:    2560(0xa00)\n  Grid Max Size:           4294967295(0xffffffff)\n  Grid Max Size per Dimension:\n    x                        4294967295(0xffffffff)\n    y                        4294967295(0xffffffff)\n    z                        4294967295(0xffffffff)\n  Max fbarriers\/Workgrp:   32\n  Packet Processor uCode:: 730\n  SDMA engine uCode::      58\n  IOMMU Support::          None\n  Pool Info:\n    Pool 1\n      Segment:                 GLOBAL; FLAGS: COARSE GRAINED\n      Size:                    8388608(0x800000) KB\n      Allocatable:             TRUE\n      Alloc Granule:           4KB\n      Alloc Alignment:         4KB\n      Accessible by all:       FALSE\n    Pool 2\n      Segment:                 GLOBAL; FLAGS:\n      Size:                    8388608(0x800000) KB\n      Allocatable:             TRUE\n      Alloc Granule:           4KB\n      Alloc Alignment:         4KB\n      Accessible by all:       FALSE\n    Pool 3\n      Segment:                 GROUP\n      Size:                    64(0x40) KB\n      Allocatable:             FALSE\n      Alloc Granule:           0KB\n      Alloc Alignment:         0KB\n      Accessible by all:       FALSE\n  ISA Info:\n    ISA 1\n      Name:                    amdgcn-amd-amdhsa--gfx803\n      Machine Models:          HSA_MACHINE_MODEL_LARGE\n      Profiles:                HSA_PROFILE_BASE\n      Default Rounding Mode:   NEAR\n      Default Rounding Mode:   NEAR\n      Fast f16:                TRUE\n      Workgroup Max Size:      1024(0x400)\n      Workgroup Max Size per Dimension:\n        x                        1024(0x400)\n        y                        1024(0x400)\n        z                        1024(0x400)\n      Grid Max Size:           4294967295(0xffffffff)\n      Grid Max Size per Dimension:\n        x                        4294967295(0xffffffff)\n        y                        4294967295(0xffffffff)\n        z                        4294967295(0xffffffff)\n      FBarrier Max Size:       32\n*** Done ***<\/pre>\n\n\n\n<p>\u4e0a\u8a18\u306e\u51fa\u529b\u306eAgent 2\u304cRX480(gfx803)\u306b\u306a\u308a\u307e\u3059\u3002GPU\u3082\u8a8d\u8b58\u3067\u304d\u3066\u3044\u308b\u3088\u3046\u3067\u3059\u306d\u3002<\/p>\n\n\n\n<p>\u6b21\u306bLLM\u3092\u52d5\u304b\u3059\u305f\u3081\u306b\u3001llama.cpp\u3092\u30b3\u30f3\u30d1\u30a4\u30eb\u3057\u307e\u3059\u3002LLAMA_HIPBLAS=1\u3092\u6307\u5b9a\u3057\u3066ROCm\u3092\u6709\u52b9\u306b\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># dnf install git make wget -y\n# cd \/work\n# git clone https:\/\/github.com\/ggerganov\/llama.cpp\n# cd \/work\/llama.cpp\n# export CC=\/opt\/rocm\/llvm\/bin\/clang\n# export CXX=\/opt\/rocm\/llvm\/bin\/clang++\n# make LLAMA_HIPBLAS=1 -j<\/pre>\n\n\n\n<p>GGUF\u5f62\u5f0f\u306e\u30e2\u30c7\u30eb\u3092\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3057\u307e\u3059\u3002\u4eca\u56de\u306f\u4ee5\u4e0b\u3092\u4f7f\u308f\u305b\u3066\u3044\u305f\u3060\u304d\u307e\u3057\u305f\u3002<br><a href=\"https:\/\/huggingface.co\/mmnga\/ELYZA-japanese-Llama-2-13b-fast-instruct-gguf\/\">https:\/\/huggingface.co\/mmnga\/ELYZA-japanese-Llama-2-13b-fast-instruct-gguf\/<\/a><br>llama2\u306e13b\u3092\u91cf\u5b50\u5316\u3057\u305f\u30e2\u30c7\u30eb\u3067\u3059\u30028GB\u5f31\u306e\u30e2\u30c7\u30eb\u306a\u306e\u3067\u307b\u307cGPU\u306b\u4e57\u305b\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u306e\u3067\u306f\u306a\u3044\u304b\u3068\u601d\u3044\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># mkdir \/work\/model\n# cd \/work\/model\n# wget https:\/\/huggingface.co\/mmnga\/ELYZA-japanese-Llama-2-13b-fast-instruct-gguf\/resolve\/main\/ELYZA-japanese-Llama-2-13b-fast-instruct-q4_K_M.gguf<\/pre>\n\n\n\n<p>41layer\u3092GPU\u30aa\u30d5\u30ed\u30fc\u30c9\u3057\u3088\u3046\u3068\u3059\u308b\u3068Out of Memory\u306b\u306a\u3063\u3066\u843d\u3061\u305f\u306e\u306739\u6307\u5b9a\u306b\u3057\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># \/work\/llama.cpp\/main -ngl 39 -m \/work\/model\/ELYZA-japanese-Llama-2-13b-fast-instruct-q4_K_M.gguf -p '[INST] &lt;&lt;SYS>>\u3042\u306a\u305f\u306f\u8aa0\u5b9f\u3067\u512a\u79c0\u306a\u65e5\u672c\u4eba\u306e\u30a2\u30b7\u30b9\u30bf\u30f3\u30c8\u3067\u3059\u3002&lt;&lt;\/SYS>>Python\u3067mysql\u304b\u3089select\u3059\u308b\u30b5\u30f3\u30d7\u30eb\u3092\u5b9f\u88c5\u3057\u3066[\/INST]'\n~~~~~~~~~~~~\u7701\u7565~~~~~~~~~~~~\nggml_cuda_init: GGML_CUDA_FORCE_MMQ:   no\nggml_cuda_init: CUDA_USE_TENSOR_CORES: yes\nggml_cuda_init: found 1 ROCm devices:\n  Device 0: AMD Radeon (TM) RX 480 Graphics, compute capability 8.0, VMM: no\nllm_load_tensors: ggml ctx size =    0.37 MiB\nllm_load_tensors: offloading 39 repeating layers to GPU\nllm_load_tensors: offloaded 39\/41 layers to GPU\nllm_load_tensors:      ROCm0 buffer size =  7090.72 MiB\nllm_load_tensors:        CPU buffer size =  7585.80 MiB\n~~~~~~~~~~~~\u7701\u7565~~~~~~~~~~~~\n [INST] &lt;&lt;SYS>>\u3042\u306a\u305f\u306f\u8aa0\u5b9f\u3067\u512a\u79c0\u306a\u65e5\u672c\u4eba\u306e\u30a2\u30b7\u30b9\u30bf\u30f3\u30c8\u3067\u3059\u3002&lt;&lt;\/SYS>>Python\u3067mysql\u304b\u3089select\u3059\u308b\u30b5\u30f3\u30d7\u30eb\u3092\u5b9f\u88c5\u3057\u3066[\/INST] mysql-connector-python\u3092pip\u3067\u30a4\u30f3\u30b9\u30c8 \u30fc\u30eb\u3057\u305f\u3089\u3001\u6b21\u306e\u3088\u3046\u306b\u5b9f\u88c5\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002\n```\nimport mysql.connector\n\ncnx = mysql.connector.connect(host='localhost', user='username', password='password')\ncursor = cnx.cursor()\n\nquery = \"SELECT * FROM table_name\"\ncursor.execute(query)\n\nresults = cursor.fetchall()\n\nfor row in results:\n    print(row)\n``` [end of text]\n\nllama_print_timings:        load time =    5032.17 ms\nllama_print_timings:      sample time =       4.97 ms \/   102 runs   (    0.05 ms per token, 20539.67 tokens per second)\nllama_print_timings: prompt eval time =   31577.39 ms \/    37 tokens (  853.44 ms per token,     1.17 tokens per second)\nllama_print_timings:        eval time =    8256.41 ms \/   101 runs   (   81.75 ms per token,    12.23 tokens per second)\nllama_print_timings:       total time =   39857.12 ms \/   138 tokens\nLog end<\/pre>\n\n\n\n<p>\u30d7\u30ed\u30f3\u30d7\u30c8\u306e\u8868\u793a\u306f12.23 tokens per second\u306a\u306e\u3067\u3001\u5b9f\u7528\u7684\u306a\u901f\u5ea6\u3060\u3068\u611f\u3058\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<p>Ryzen 2400G\u3067\u540c\u69d8\u306e\u30e2\u30c7\u30eb\u3067CPU\u3067\u5b9f\u884c\u3057\u305f\u5834\u5408\u30013.86 tokens per second\u3060\u3063\u305f\u306e\u3067\u3001RX480\u306f\u306f3\u500d\u901f\u3044\u3067\u3059\u3002\u305f\u3060\u3001load time\u306fGPU\u3092\u4f7f\u3046\u30b1\u30fc\u30b9\u3067\u306f\u9577\u304f\u306a\u308b\u3088\u3046\u3067\u3059\u3002<\/p>\n\n\n\n<p>\u305b\u3063\u304b\u304f\u306a\u306e\u3067\u3001\u5168\u90e8VRAM\u306b\u4e57\u305b\u3089\u308c\u308b\u306e\u304c\u671f\u5f85\u3067\u304d\u308b7B\u306e\u30e2\u30c7\u30eb\u3082\u8a66\u3057\u3066\u307f\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># cd \/work\/model\n# wget https:\/\/huggingface.co\/mmnga\/ELYZA-japanese-Llama-2-7b-instruct-gguf\/resolve\/main\/ELYZA-japanese-Llama-2-7b-instruct-q4_K_M.gguf\n# \/work\/llama.cpp\/main -ngl 33 -m \/work\/model\/ELYZA-japanese-Llama-2-7b-instruct-q4_K_M.gguf -p '[INST] &lt;&lt;SYS>>\u3042\u306a\u305f\u306f\u8aa0\u5b9f\u3067\u512a\u79c0\u306a\u65e5\u672c\u4eba\u306e\u30a2\u30b7\u30b9\u30bf\u30f3\u30c8\u3067\u3059\u3002&lt;&lt;\/SYS>>Python\u3067mysql\u304b\u3089select\u3059\u308b\u30b5\u30f3\u30d7\u30eb\u3092\u5b9f\u88c5\u3057\u3066[\/INST]'\n~~~~~~~~~~~~\u7701\u7565~~~~~~~~~~~~\nggml_cuda_init: GGML_CUDA_FORCE_MMQ:   no\nggml_cuda_init: CUDA_USE_TENSOR_CORES: yes\nggml_cuda_init: found 1 ROCm devices:\n  Device 0: AMD Radeon (TM) RX 480 Graphics, compute capability 8.0, VMM: no\nllm_load_tensors: ggml ctx size =    0.30 MiB\nllm_load_tensors: offloading 32 repeating layers to GPU\nllm_load_tensors: offloading non-repeating layers to GPU\nllm_load_tensors: offloaded 33\/33 layers to GPU\nllm_load_tensors:      ROCm0 buffer size =  3820.94 MiB\nllm_load_tensors:        CPU buffer size =    70.31 MiB\n~~~~~~~~~~~~\u7701\u7565~~~~~~~~~~~~\n\n [INST] &lt;&lt;SYS>>\u3042\u306a\u305f\u306f\u8aa0\u5b9f\u3067\u512a\u79c0\u306a\u65e5\u672c\u4eba\u306e\u30a2\u30b7\u30b9\u30bf\u30f3\u30c8\u3067\u3059\u3002&lt;&lt;\/SYS>>Python\u3067mysql\u304b\u3089select\u3059\u308b\u30b5\u30f3\u30d7\u30eb\u3092\u5b9f\u88c5\u3057\u3066[\/INST]  ```python\nimport mysql.connector\n\n# \u30af\u30e9\u30a4\u30a2\u30f3\u30c8\u30e9\u30a4\u30d6\u30e9\u30ea\u306e\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\nmysql.connector\/connectors\/pandas-mysql.pyx\n\n# \u63a5\u7d9a\u60c5\u5831\u306e\u8a2d\u5b9a\nhost = 'localhost'\nusername = 'root'\npassword = ''\ndbname = 'test'\n\n# \u63a5\u7d9a\ncnx = mysql.connector.connect(host=host, user=username, passwd=password,\n                                         database=dbname)\n\n# \u30c7\u30fc\u30bf\u306e\u53d6\u5f97\ncursor = cnx.cursor()\n\n# \u30b9\u30c6\u30fc\u30c8\u30e1\u30f3\u30c8\ncursor.execute(\"SELECT * FROM table1\")\n\n# \u30c7\u30fc\u30bf\u3092\u8aad\u307f\u8fbc\u3080\ndf = pd.io.sql(cursor, index_col=None)\n\n# \u30af\u30ed\u30fc\u30ba\ncnx.close()\n\n# \u30c7\u30fc\u30bf\u3092\u8868\u793a\nprint(df)\n``` [end of text]\n\nllama_print_timings:        load time =    3086.38 ms\nllama_print_timings:      sample time =       7.60 ms \/   228 runs   (    0.03 ms per token, 30000.00 tokens per second)\nllama_print_timings: prompt eval time =   15435.83 ms \/    61 tokens (  253.05 ms per token,     3.95 tokens per second)\nllama_print_timings:        eval time =   10469.52 ms \/   227 runs   (   46.12 ms per token,    21.68 tokens per second)\nllama_print_timings:       total time =   25941.17 ms \/   288 tokens\nLog end<\/pre>\n\n\n\n<p>RX480\u306721.68 tokens per second\u3067\u3057\u305f\u3002Web\u306eAI\u7cfb\u306e\u30b5\u30fc\u30d3\u30b9\u3067\u306e\u901f\u5ea6\u3068\u905c\u8272\u306a\u3044\u3067\u3059\u3002<\/p>\n\n\n\n<p>Ryzen 2400G\u3067\u306f7.01 tokens per second\u3060\u3063\u305f\u306e\u3067\u3053\u3061\u3089\u30823\u500d\u3067\u3059\u306d\u3002<br>\u6b8b\u308a1layer\u306a\u3089\u3059\u3079\u3066VRAM\u306b\u8f09\u305b\u304d\u3063\u3066\u3082\u305d\u3053\u307e\u3067\u5b9f\u884c\u901f\u5ea6\u306b\u306f\u5f71\u97ff\u3067\u306a\u3044\u3088\u3046\u3067\u3059\u3002<\/p>\n\n\n\n<p>\u7b2c\u4e00\u4e16\u4ee3\u306eRyzen\u306fWindows11\u306b\u30a2\u30c3\u30d7\u30b0\u30ec\u30fc\u30c9\u3067\u304d\u306a\u3044\u306e\u3067\u3001\u3069\u3046\u3059\u308b\u304b\u3068\u601d\u3063\u3066\u3044\u307e\u3057\u305f\u304c\u3001\u3053\u308c\u306a\u3089\u30ed\u30fc\u30ab\u30eb\u306eAIChat\u74b0\u5883\u3068\u3057\u3066\u4f59\u751f\u304c\u9001\u308c\u305d\u3046\u3067\u3059\u3002<\/p>\n\n\n\n<p>RX480\u304cFP16\u30675.834 TFLOPS \u30e1\u30e2\u30ea\u5e2f\u57df256GB\/s\u3068\u305d\u3053\u307e\u3067\u30cf\u30a4\u30b9\u30da\u30c3\u30af\u3067\u306f\u306a\u3044\u74b0\u5883\u3067\u3053\u308c\u306a\u306e\u3067\u3001\u4e2d\u53e4\u3067\u5b89\u304f\u306b\u5165\u624b\u3067\u304d\u308bVEGA56\uff08FP16 21.09 TFLOPS \u30e1\u30e2\u30ea\u5e2f\u57df409.6GB\/s)\u3042\u305f\u308a\u3092\u5165\u624b\u3059\u308c\u3070\u9ad8\u901f\u5316\u3082\u72d9\u3048\u305d\u3046\u3067\u3059\u3002<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Ryzen Radeon LLM Rocky Linux ChatGPT ROCm RHEL<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[19,25],"tags":[],"class_list":["post-683","post","type-post","status-publish","format-standard","hentry","category-linux","category-programming"],"_links":{"self":[{"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/posts\/683","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=683"}],"version-history":[{"count":5,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/posts\/683\/revisions"}],"predecessor-version":[{"id":705,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/posts\/683\/revisions\/705"}],"wp:attachment":[{"href":"https:\/\/blog.gurees.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=683"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=683"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=683"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}