黔东南网站设计公司,网站维护包括,全屋定制哪个网站好,wordpress 种子搜索引擎本文介绍了清洁安装的Ubuntu Server 22.04 LTS安装NVIDIA显卡驱动、CUDA 12.1、cuDNN的方法及ChatGLM3、百川2、FastChat等大语言模型的部署使用方法。
安装NVIDIA驱动
禁用nouveau
sudo vi /etc/modprobe.d/blacklist.conf尾部追加一行
blacklist nouveau执行并重启系统
…本文介绍了清洁安装的Ubuntu Server 22.04 LTS安装NVIDIA显卡驱动、CUDA 12.1、cuDNN的方法及ChatGLM3、百川2、FastChat等大语言模型的部署使用方法。
安装NVIDIA驱动
禁用nouveau
sudo vi /etc/modprobe.d/blacklist.conf尾部追加一行
blacklist nouveau执行并重启系统
sudo update-initramfs -u
reboot检查nouveau是否关闭成功应当无输出
lsmod | grep nouveau安装535驱动
sudo apt update
sudo apt install nvidia-driver-535 -y当然也可以查询推荐驱动选择recommended的版本安装
ubuntu-drivers devices装完后执行nvidia-smi验证应该有正确的输出。
安装CUDA12.1
下载并执行
wget https://developer.download.nvidia.com/compute/cuda/12.1.0/local_installers/cuda_12.1.0_530.30.02_linux.run
chmod x cuda_12.1.0_530.30.02_linux.run
sh cuda_12.1.0_530.30.02_linux.run执行后要等一会加载然后进入交互式界面按如下步骤操作
提示已经存在驱动…选择continue阅读并接受协议输入accept回车上下光标选中- [X] Driver列按空格以取消勾选驱动再选择Install回车等待安装完成
编辑环境变量
vi ~/.bashrc
尾部追加
export PATH/usr/local/cuda-12.1/bin:$PATH
export LD_LIBRARY_PATH/usr/local/cuda-12.1/lib64:$LD_LIBRARY_PATH
执行
source ~/.bashrc 验证
nvcc --version
systemctl status nvidia-persistenced均应输出有效信息。
安装NVIDIA-Docker可选
官网文档链接https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
添加源并安装
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \sed s#deb https://#deb [signed-by/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g | \sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.listsudo apt-get update
sudo apt-get install -y nvidia-container-toolkit nvidia-container-runtime在安装好Docker后执行如下操作以配置Docker
sudo nvidia-ctk runtime configure --runtimedocker
sudo systemctl restart docker执行以下命令验证
docker run --gpus all -it --name nvtest nvidia/cuda:12.3.1-base-ubuntu22.04 /bin/sh
nvidia-smi附 使用所有GPU docker run --gpus all nvidia/cuda:12.3.1-base-ubuntu22.04 nvidia-smi 使用两个GPU docker run --gpus 2 nvidia/cuda:12.3.1-base-ubuntu22.04 nvidia-smi 指定GPU运行 docker run --gpus device1,2 nvidia/cuda:12.3.1-base-ubuntu22.04 nvidia-smi docker run --gpus deviceUUID-ABCDEF,1 nvidia/cuda:12.3.1-base-ubuntu22.04 nvidia-smi
安装cuDNN可选
官网文档链接https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#installlinux-deb
记得安装zlib
sudo apt-get install zlib1g安装Miniconda
下载并安装miniconda
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Miniconda3-latest-Linux-x86_64.sh
source ~/.bashrcconda和pip换源
conda config --set show_channel_urls yes
vi ~/.condarcchannels:- defaults
show_channel_urls: true
default_channels:- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmsys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudbioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmenpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudpytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudpytorch-lts: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudsimpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/clouddeepmodeling: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/
ssl_verify: falsepip config set global.index-url https://mirrors.aliyun.com/pypi/simple/使用ModelScope下载模型到本地
新建一个下载模型的脚本download_model.py
import sys
from modelscope import snapshot_downloadif len(sys.argv) 2:print(Usage: python download_model.py model_name)sys.exit(1)model_name sys.argv[1]
model_dir snapshot_download(model_name, revisionmaster)
print(model_dir)再新建一个shell脚本dl_model.sh用于拉起模型下载脚本
#!/bin/bashif [ $# -ne 1 ]; thenecho Usage: model_nameexit 1
fipython download_model.py $1设置代理脚本加速github等
设置代理setproxy
#!/bin/sh# for terminal
export proxyserveraddr192.168.114.114
export proxyserverport7890
export HTTP_PROXYhttp://$proxyserveraddr:$proxyserverport/
export HTTPS_PROXYhttps://$proxyserveraddr:$proxyserverport/
export FTP_PROXYftp://$proxyserveraddr:$proxyserverport/
export SOCKS_PROXYsocks://$proxyserveraddr:$proxyserverport/
export NO_PROXYlocalhost,127.0.0.1,localaddress,.localdomain.com,可以新增你想过滤的ip段;
export http_proxyhttp://$proxyserveraddr:$proxyserverport/
export https_proxyhttps://$proxyserveraddr:$proxyserverport/
export ftp_proxyftp://$proxyserveraddr:$proxyserverport/
export socks_proxysocks://$proxyserveraddr:$proxyserverport/
export no_proxylocalhost,127.0.0.1,localaddress,.localdomain.com,可以新增你想过滤的ip段;# for apt-get
cat -EOF| sudo tee /etc/apt/apt.conf
Acquire::http::proxy http://$proxyserveraddr:$proxyserverport/;
Acquire::https::proxy https://$proxyserveraddr:$proxyserverport/;
Acquire::ftp::proxy ftp://$proxyserveraddr:$proxyserverport/;
Acquire::socks::proxy socks://$proxyserveraddr:$proxyserverport/;
EOF取消代理unsetproxy
#!/bin/sh
unset proxyserveraddr
unset proxyserverport
unset HTTP_PROXY
unset HTTPS_PROXY
unset FTP_PROXY
unset SOCKS_PROXY
unset NO_PROXY
unset http_proxy
unset https_proxy
unset ftp_proxy
unset socks_proxy
unset no_proxy
gsettings reset org.gnome.system.proxy ignore-hosts
echo -n |sudo tee /etc/apt/apt.conf使用方法
source setproxy
source unsetproxy部署ChatGLM3
创建虚拟环境
conda create -n chatglm python3.11激活开搞
conda activate chatglm
git clone https://github.com/THUDM/ChatGLM3.git
pip install modelscope
pip install -r requirements.txt安装pytorch
conda install pytorch torchvision torchaudio pytorch-cuda12.1 -c pytorch -c nvidia下载模型
dl_model.sh ZhipuAI/chatglm3-6b下载的模型路径为/root/.cache/modelscope/hub/ZhipuAI/chatglm3-6b
编写composite_demo启动脚本openai_api_demo等同理
cd /root/ChatGLM3/composite_demo
vi start.sh以下是start.sh的内容
export MODEL_PATH/root/.cache/modelscope/hub/ZhipuAI/chatglm3-6b
streamlit run main.py如果要使用composite_demo的code interpreter首次启动前记得执行
ipython kernel install --name chatglm3 --user部署百川2
前面的虚拟环境准备、代码库拉取、模型下载等第1~4节和ChatGLM3差不多。
百川2的git仓库地址是https://github.com/baichuan-inc/Baichuan2.git
魔搭的模型名是baichuan-inc/Baichuan2-7B-Chat或13B
配置修改启动脚本的init_model函数
vi Baichuan2/web_demo.pyweb_demo.py的修改后函数如下
st.cache_resource
def init_model():model AutoModelForCausalLM.from_pretrained(/root/.cache/modelscope/hub/baichuan-inc/Baichuan2-7B-Chat,torch_dtypetorch.float16,device_mapauto,trust_remote_codeTrue)model.generation_config GenerationConfig.from_pretrained(/root/.cache/modelscope/hub/baichuan-inc/Baichuan2-7B-Chat)tokenizer AutoTokenizer.from_pretrained(/root/.cache/modelscope/hub/baichuan-inc/Baichuan2-7B-Chat,use_fastFalse,trust_remote_codeTrue)return model, tokenizer如果要以int8量化模式运行在torch_dtypetorch.float16,后添加一行load_in_8bitTrue,即可。
执行streamlit run web_demo.py
搭建FastChat服务
搭建FastChat环境
conda create -n fastchat python3.11
conda activate fastchat
git clone https://github.com/lm-sys/FastChat.git
cd FastChat
pip3 install --upgrade pip
pip3 install -e .[model_worker,webui]写一组FastChat启动脚本 fastchat.sh使用anaconda就把miniconda3换成anaconda3
#!/bin/bash
source ~/miniconda3/etc/profile.d/conda.sh
conda activate fastchat
if [ $? -ne 0 ]; thenecho Failed to activate Anaconda environment fastchat.exit 1
fi
/root/fastchat-process/main.sh $创建相关文件和文件夹
mkdir ~/fastchat-process
touch /root/fastchat-process/controller.log
touch /root/fastchat-process/gradio_web_server.log
touch /root/fastchat-process/model_worker.log
touch /root/fastchat-process/openai_api_server.log创建fastchat-process/main.sh
#!/bin/bashsource ~/anaconda3/etc/profile.d/conda.sh
conda activate fastchatwait_for_startup() {while ! grep -q Application startup complete (tail -n 10 -f $1); dosleep 1done
}python3 -m fastchat.serve.controller /root/fastchat-process/controller.log
echo Controller is starting...
wait_for_startup /root/fastchat-process/controller.logpython3 -m fastchat.serve.model_worker --model-path $1 /root/fastchat-process/model_worker.log
echo Model worker is starting with model path $1...
wait_for_startup /root/fastchat-process/model_worker.logpython3 -m fastchat.serve.openai_api_server --host localhost --port $2 /root/fastchat-process/openai_api_server.log
echo OpenAI API server is starting on localhost:$2...
wait_for_startup /root/fastchat-process/openai_api_server.logpython3 -m fastchat.serve.gradio_web_server /root/fastchat-process/gradio_web_server.log
echo Gradio web server is starting...
wait_for_startup /root/fastchat-process/gradio_web_server.logecho All services have been started and are ready.用法./fastchat.sh /root/Yi-6B-Chat 8000
这将启动一个OPENAI_API端口为8000的服务和一个gradio web服务。具体日志可以在对应log中找到。
