{"id":688,"date":"2024-06-02T16:57:58","date_gmt":"2024-06-02T07:57:58","guid":{"rendered":"https:\/\/blog.gurees.net\/?p=688"},"modified":"2024-06-02T16:57:58","modified_gmt":"2024-06-02T07:57:58","slug":"rocky-linux%e3%81%a8radeon-rx-480%e3%81%a7rocm%e3%82%92%e4%bd%bf%e3%81%a3%e3%81%a6%e3%83%ad%e3%83%bc%e3%82%ab%e3%83%abllm%e3%82%92%e5%8b%95%e3%81%8b%e3%81%97%e3%81%a6%e3%81%bf%e3%82%8bgui%e7%b7%a8","status":"publish","type":"post","link":"https:\/\/blog.gurees.net\/?p=688","title":{"rendered":"Rocky Linux\u3068Radeon RX 480\u3067ROCm\u3092\u4f7f\u3063\u3066\u30ed\u30fc\u30ab\u30ebLLM\u3092\u52d5\u304b\u3057\u3066\u307f\u308b(GUI\u7de8)"},"content":{"rendered":"\n

\u524d\u56de\u3001llama.cpp\u3092\u4f7f\u3063\u3066LLM\u3092\u30ed\u30fc\u30ab\u30eb\u3067\u52d5\u304b\u3059\u3053\u3068\u304c\u3067\u304d\u305f\u306e\u3067\u3001\u30b5\u30fc\u30d0\u6a5f\u80fd\u3092\u4f7f\u3063\u3066API\u30b5\u30fc\u30d0\u3092\u8d77\u52d5\u3057\u3001\u30d6\u30e9\u30a6\u30b6\u304b\u3089\u4f7f\u3063\u3066\u307f\u3088\u3046\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n\n\n\n\n\n\n\n

llama.cpp\u306eserver\u3092\u4f7f\u3063\u3066\u3001API\u30b5\u30fc\u30d0\u3092\u8d77\u52d5\u3057\u307e\u3059\u3002<\/p>\n\n\n\n

# \/work\/llama.cpp\/server -ngl 39 -m \/work\/model\/ELYZA-japanese-Llama-2-7b-instruct-q4_K_M.gguf --host 0.0.0.0 --port 8080<\/pre>\n\n\n\n
curl\u3067API\u306bPOST\u3057\u3066\u52d5\u4f5c\u78ba\u8a8d\u3057\u307e\u3059\u3002<\/p>\n\n\n\n
# curl --request POST\\\n     --url http:\/\/localhost:8080\/completion\\\n     --header \"Content-Type: application\/json\"\\\n     --data '{\"prompt\": \"[INST] <<SYS>>\u3042\u306a\u305f\u306f\u8aa0\u5b9f\u3067\u512a\u79c0\u306a\u65e5\u672c\u4eba\u306e\u30a2\u30b7\u30b9\u30bf\u30f3\u30c8\u3067\u3059\u3002<<\/SYS>>Python\u3067mysql\u304b \u3089select\u3059\u308b\u30b5\u30f3\u30d7\u30eb\u3092\u5b9f\u88c5\u3057\u3066[\/INST]\",\"n_predict\": 256}'\n\n{\"content\":\" Python\u3067MySQL\u304b\u3089\u306e\u30c7\u30fc\u30bf\u9078\u629e\u3092\u884c\u3046\u305f\u3081\u306e\u30b5\u30f3\u30d7\u30eb\u30b3\u30fc\u30c9\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002\\n\\n\u307e\u305a\u3001Python\u3067MySQL\u306b\u63a5\u7d9a\u3059\u308b\u30e2\u30b8\u30e5\u30fc\u30eb\u3092\u6c42\u3081\u307e\u3059\u3002\u3053\u3053\u3067\u306fPyMySQL\u30e2\u30b8\u30e5\u30fc\u30eb\u3092\u4f7f\u7528\u3057\u307e\u3059\u3002\\n```\\nimport pymysql\\n```\\n\u6b21\u306b\u3001MySQL\u306e\u30b5\u30fc\u30d0\u30fc\u60c5\u5831\u3001\u30e6\u30fc\u30b6\u30fc \u540d\u3001\u30d1\u30b9\u30ef\u30fc\u30c9\u3001\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u540d\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002\\n```\\nserver = \\\"localhost\\\"\\nuser = \\\"your_username\\\"\\npassword = \\\"your_password\\\"\\ndb = \\\"your_database\\\"\\n```\\n\u6b21\u306b\u3001PyMySQL\u306eConnect\u95a2\u6570\u3092\u547c\u3073\u51fa\u3057\u3001\u63a5\u7d9a\u60c5\u5831\u3092\u6e21\u3057\u3066MySQL\u306b\u63a5\u7d9a\u3057\u307e\u3059\u3002\\n```\\nconn = pymysql.connect(host=server, user=user, passwd=password, db=db)\\n```\\n\u6b21 \u306b\u3001PyMySQL\u306eCurser\u3092\u958b\u304d\u3001SQL\u6587\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002\\n```\\ncursor = conn.cursor()\\nquery = \\\"SELECT * FROM your_table\\\"\\ncursor.execute(query)\\n```\\n\u6b21\u306b\u3001PyMySQL\u306efetchall()\u95a2\u6570\u3092\u547c\u3073\u51fa\u3057\u3001\u30c7\u30fc\u30bf\u30bb\u30ec\u30af\u30c8\u306e\u7d50\u679c\u3092\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306e\u884c\u3068\u3057\u3066\u53d6\u5f97\u3057\u307e\u3059\u3002\\n```\\nresult = cursor.fetchall()\\n```\\n\u6b21\u306b\u3001PyMySQL\u306ecommit()\u95a2\u6570\u3092\u547c\u3073\u51fa\u3057\u3001 \u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306b\u30b3\u30df\u30c3\u30c8\",\"id_slot\":0,\"stop\":true,\"model\":\"\/work\/model\/ELYZA-japanese-Llama-2-13b-fast-instruct-q4_K_M.gguf\",\"tokens_predicted\":256,\"tokens_evaluated\":37,\"generation_settings\":{\"n_ctx\":512,\"n_predict\":-1,\"model\":\"\/work\/model\/ELYZA-japanese-Llama-2-13b-fast-instruct-q4_K_M.gguf\",\"seed\":4294967295,\"temperature\":0.800000011920929,\"dynatemp_range\":0.0,\"dynatemp_exponent\":1.0,\"top_k\":40,\"top_p\":0.949999988079071,\"min_p\":0.05000000074505806,\"tfs_z\":1.0,\"typical_p\":1.0,\"repeat_last_n\":64,\"repeat_penalty\":1.0,\"presence_penalty\":0.0,\"frequency_penalty\":0.0,\"penalty_prompt_tokens\":[],\"use_penalty_prompt_tokens\":false,\"mirostat\":0,\"mirostat_tau\":5.0,\"mirostat_eta\":0.10000000149011612,\"penalize_nl\":false,\"stop\":[],\"n_keep\":0,\"n_discard\":0,\"ignore_eos\":false,\"stream\":false,\"logit_bias\":[],\"n_probs\":0,\"min_keep\":0,\"grammar\":\"\",\"samplers\":[\"top_k\",\"tfs_z\",\"typical_p\",\"top_p\",\"min_p\",\"temperature\"]},\"prompt\":\"[INST] <<SYS>>\u3042\u306a\u305f\u306f\u8aa0\u5b9f\u3067\u512a\u79c0\u306a\u65e5\u672c \u4eba\u306e\u30a2\u30b7\u30b9\u30bf\u30f3\u30c8\u3067\u3059\u3002<<\/SYS>>Python\u3067mysql\u304b\u3089select\u3059\u308b\u30b5\u30f3\u30d7\u30eb\u3092\u5b9f\u88c5\u3057\u3066[\/INST]\",\"truncated\":false,\"stopped_eos\":false,\"stopped_word\":false,\"stopped_limit\":true,\"stopping_word\":\"\",\"tokens_cached\":292,\"timings\":{\"prompt_n\":37,\"prompt_ms\":31625.747,\"prompt_per_token_ms\":854.7499189189189,\"prompt_per_second\":1.1699328398472295,\"predicted_n\":256,\"predicted_ms\":22570.624,\"predicted_per_token_ms\":88.1665,\"predicted_per_second\":11.342176450239036}}redicted_per_token_ms\":83.9495546875,\"predicted_per_second\":11.911915479748208}}<\/pre>\n\n\n\n
WebAPI\u3067\u3082\u540c\u69d8\u306b\u5fdc\u7b54\u304c\u8fd4\u3063\u3066\u304d\u307e\u3059\u306d\u3002<\/p>\n\n\n\n
WebAPI\u3068\u304a\u8a71\u3057\u3059\u308b\u306e\u306bcrul\u3092\u4f7f\u3063\u3066\u3044\u3066\u306f\u30b3\u30de\u30f3\u30c9\u30e9\u30a4\u30f3\u5b9f\u884c\u3068\u5909\u308f\u3089\u306a\u3044\u306e\u3067Web\u30b5\u30fc\u30d0\u3092\u7acb\u3066\u307e\u3059\u3002
\u7d50\u69cb\u3044\u308d\u3093\u306aUI\u304c\u3042\u308b\u307f\u305f\u3044\u3067\u3059\u304c\u3001\u3068\u308a\u3042\u3048\u305a\u30b7\u30f3\u30d7\u30eb\u306b\u884c\u304d\u307e\u3059\u3002<\/p>\n\n\n\n
# dnf install -y httpd<\/pre>\n\n\n\n
UI\u306e\u90e8\u5206\u3067\u3059\u3002\/var\/www\/html\/index.html\u3068\u3057\u3066\u4fdd\u5b58\u3057\u307e\u3059\u3002<\/p>\n\n\n\n
<!DOCTYPE html>\n<html lang=\"ja\">\n<head>\n    <meta charset=\"UTF-8\">\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n    <script src=\".\/script.js\" defer><\/script>\n    <title>Chat with llama.cpp<\/title>\n    <style>\n        body { \n            font-family: Arial, sans-serif; \n            background-color: #f4f4f9; \n            margin: 0; \n            padding: 0; \n            display: flex; \n            justify-content: center; \n            align-items: center; \n            height: 100vh; \n        }\n        .chat-container { \n            width: 80%; \n            max-width: 1200px; \n            background-color: #fff; \n            margin: auto; \n            padding: 20px; \n            border: 1px solid #ddd; \n            border-radius: 10px; \n            box-shadow: 0 4px 8px rgba(0, 0, 0, 0.1); \n            display: flex; \n            flex-direction: column; \n            height: calc(100vh - 40px); \n        }\n        .chat-box { \n            flex: 1; \n            width: 100%; \n            border: 1px solid #ddd; \n            padding: 10px; \n            overflow-y: scroll; \n            border-radius: 5px; \n            background-color: #fafafa; \n            margin-bottom: 10px; \n        }\n        .input-box-container { \n            display: flex; \n        }\n        .input-box { \n            flex: 1; \n            padding: 10px; \n            border: 1px solid #ddd; \n            border-radius: 5px; \n            margin-right: 10px; \n            resize: none; \n            height: 100px; \n        }\n        .send-button { \n            padding: 10px 20px; \n            border: none; \n            background-color: #4CAF50; \n            color: white; \n            border-radius: 5px; \n            cursor: pointer; \n            transition: background-color 0.3s; \n        }\n        .send-button:disabled {\n            background-color: #9E9E9E;\n            cursor: not-allowed;\n        }\n        .send-button:hover:enabled { \n            background-color: #45a049; \n        }\n        .message { \n            margin: 5px 0; \n            padding: 10px; \n            border-radius: 5px; \n        }\n        .user { \n            background-color: #d1e7dd; \n            align-self: flex-end; \n            text-align: right; \n        }\n        .bot { \n            background-color: #f8d7da; \n            align-self: flex-start; \n            text-align: left; \n        }\n    <\/style>\n<\/head>\n<body>\n    <div class=\"chat-container\">\n        <h1>Chat with llama.cpp<\/h1>\n        <div class=\"chat-box\" id=\"chat-box\"><\/div>\n        <div class=\"input-box-container\">\n            <textarea id=\"user-input\" class=\"input-box\" placeholder=\"Type a message...\"><\/textarea>\n            <button id=\"send-button\" class=\"send-button\" onclick=\"sendMessage()\">Send<\/button>\n        <\/div>\n    <\/div>\n<\/body>\n<\/html>\n<\/pre>\n\n\n\n
API\u3068\u3084\u308a\u53d6\u308a\u3059\u308bscript.js\u3067\u3059\u3002<\/p>\n\n\n\n
let botThinking = false;\nlet botToking = false;\n\nfunction formatCodeBlock(text) {\n    const codeRegex = \/```([\\s\\S]+?)```\/g;\n    return text.replace(codeRegex, (match, p1) => {\n        const codeElement = document.createElement('pre');\n        const codeContent = document.createElement('code');\n        codeContent.textContent = p1.trim();\n        codeElement.appendChild(codeContent);\n        return codeElement.outerHTML;\n    });\n}\n\nfunction formatMessageText(text) {\n    return text.replace(\/\\n\/g, '<br>');\n}\n\nfunction updateSendButtonState() {\n    const sendButton = document.getElementById('send-button');\n    sendButton.disabled = botThinking || botToking;\n}\n\nasync function sendMessage() {\n    const userInput = document.getElementById('user-input').value;\n    if (!userInput) return;\n\n    const chatBox = document.getElementById('chat-box');\n    appendMessage('You: ' + formatMessageText(userInput), 'user');\n\n    const prompt = `[INST] <<SYS>>\u3042\u306a\u305f\u306f\u8aa0\u5b9f\u3067\u512a\u79c0\u306a\u65e5\u672c\u4eba\u306e\u30a2\u30b7\u30b9\u30bf\u30f3\u30c8\u3067\u3059\u3002<<\/SYS> ${userInput}[\/INST]`;\n\n    const botMessage = appendMessage('Bot: \u30fb\u30fb\u30fb', 'bot');\n\n    botThinking = true;\n    updateSendButtonState();\n\n    try {\n        const response = await fetch('http:\/\/127.0.0.1:8080\/completions', {\n            method: 'POST',\n            headers: {\n                'Content-Type': 'application\/json',\n                'Authorization': 'Bearer YOUR_API_KEY'\n            },\n            body: JSON.stringify({\n                prompt: prompt,\n                max_tokens: 256,\n                stream: true\n            })\n        });\n\n        const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();\n\n        while (true) {\n            const { value, done } = await reader.read();\n            if (done) break;\n\n            const lines = value.split('\\n').filter(line => line.trim() !== '');\n            for (const line of lines) {\n                if (line.startsWith('data:')) {\n                    const json = line.substring(5).trim();\n                    if (json !== '[DONE]') {\n                        try {\n                            const parsed = JSON.parse(json);\n                            if (parsed.content) {\n                                if (botThinking) {\n                                    botMessage.innerHTML = 'Bot: ';\n                                    botThinking = false;\n                                    botToking = true;\n                                    updateSendButtonState();\n                                }\n\n                                botMessage.innerHTML += formatCodeBlock(formatMessageText(parsed.content));\n                                chatBox.scrollTop = chatBox.scrollHeight;\n                            }\n                            if (parsed.stop || parsed.content == \"\") {\n                                reader.cancel();\n                                botToking = false;\n                                updateSendButtonState();\n                                break;\n                            }\n                        } catch (e) {\n                            console.error('Failed to parse JSON:', e);\n                        }\n                    }\n                }\n            }\n        }\n    } catch (error) {\n        botMessage.innerHTML = 'Bot: \u30a8\u30e9\u30fc\u304c\u767a\u751f\u3057\u307e\u3057\u305f\u3002';\n        console.error('Error:', error);\n    }\n\n    document.getElementById('user-input').value = '';\n    chatBox.scrollTop = chatBox.scrollHeight;\n}\n\nfunction appendMessage(text, type) {\n    const chatBox = document.getElementById('chat-box');\n    const message = document.createElement('div');\n    message.classList.add('message', type);\n    message.innerHTML = text;\n    chatBox.appendChild(message);\n    chatBox.scrollTop = chatBox.scrollHeight;\n    return message;\n}\n\ndocument.getElementById('user-input').addEventListener('keydown', function(event) {\n    if (event.key === 'Enter' && !event.shiftKey) {\n        event.preventDefault();\n        sendMessage();\n    }\n});<\/pre>\n\n\n\n
\u30d5\u30a1\u30a4\u30eb\u3092\u7f6e\u3044\u305f\u3089httpd\u3092\u8d77\u52d5\u3057\u307e\u3059\u3002<\/p>\n\n\n\n
# systemctl enable --now httpd<\/pre>\n\n\n\n
\u30d6\u30e9\u30a6\u30b6\u3067\u958b\u304f\u3068\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\u904e\u53bb\u306e\u4f1a\u8a71\u3092\u8003\u616e\u3057\u305f\u308a\u306f\u3057\u3066\u3044\u306a\u3044\u306e\u3067\u3001\u4f1a\u8a71\u3068\u3044\u3046\u611f\u3058\u3067\u306f\u306a\u3044\u3067\u3059\u304c\u3001\u30b3\u30de\u30f3\u30c9\u3092\u6253\u3064\u3088\u308a\u4f7f\u3044\u52dd\u624b\u826f\u3044\u306f\u305a\u3067\u3059\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"
\u524d\u56de\u3001llama.cpp\u3092\u4f7f\u3063\u3066LLM\u3092\u30ed\u30fc\u30ab\u30eb\u3067\u52d5\u304b\u3059\u3053\u3068\u304c\u3067\u304d\u305f\u306e\u3067\u3001\u30b5\u30fc\u30d0\u6a5f\u80fd\u3092\u4f7f\u3063\u3066API\u30b5\u30fc\u30d0\u3092\u8d77\u52d5\u3057\u3001\u30d6\u30e9\u30a6\u30b6\u304b\u3089\u4f7f\u3063\u3066\u307f\u3088\u3046\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[37,19,25,38],"tags":[],"class_list":["post-688","post","type-post","status-publish","format-standard","hentry","category-javascript","category-linux","category-programming","category-38"],"_links":{"self":[{"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/posts\/688","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=688"}],"version-history":[{"count":3,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/posts\/688\/revisions"}],"predecessor-version":[{"id":692,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=\/wp\/v2\/posts\/688\/revisions\/692"}],"wp:attachment":[{"href":"https:\/\/blog.gurees.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=688"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=688"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.gurees.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=688"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}