JZTXT

LLM部署

发布时间 2023-08-10 13:07:50作者: nanmi

LLM部署

如何从huggingface上下载模型

安装git-lfs

curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
git lfs install

从huggingface上下载模型

#请确保已安装git-lfs(https://git-lfs.com)
git-lfs install
#以llama-7b为例
git clone https://huggingface.co/huggyllama/llama-7b

如果你想在没有大文件的情况下进行克隆，只需它们的指针在git克隆之前加上以下环境变量：

GIT_LFS_SKIP_SMUDGE=1

通过pyllama下载
安装pyllama, pip install pyllama -U
下载7B的模型, python -m llama.download --model_size 7B

optimum-cli export onnx [-h] -m MODEL [--task TASK] [--opset OPSET] [--device DEVICE] [--fp16] [--optimize {O1,O2,O3,O4}] [--monolith] [--no-post-process] [--framework {pt,tf}]
                               [--atol ATOL] [--cache_dir CACHE_DIR] [--trust-remote-code] [--pad_token_id PAD_TOKEN_ID] [--batch_size BATCH_SIZE] [--sequence_length SEQUENCE_LENGTH]
                               [--num_choices NUM_CHOICES] [--width WIDTH] [--height HEIGHT] [--num_channels NUM_CHANNELS] [--feature_size FEATURE_SIZE] [--nb_max_frames NB_MAX_FRAMES]
                               [--audio_sequence_length AUDIO_SEQUENCE_LENGTH] [--point_batch_size POINT_BATCH_SIZE] [--nb_points_per_image NB_POINTS_PER_IMAGE]
                               output

reference

https://github.com/karpathy/llama2.c
https://github.com/THUDM/CodeGeeX2
https://github.com/liltom-eth/llama2-webui
https://github.com/Alpha-VLLM/LLaMA2-Accessory
https://github.com/FlagAlpha/Llama2-Chinese
https://github.com/LinkSoul-AI/Chinese-Llama-2-7b
https://github.com/ymcui/Chinese-LLaMA-Alpaca-2

https://github.com/jmorganca/ollama