近期需要做相關(guān)的一些驗(yàn)證工作想把相關(guān)的一些內(nèi)容記錄下來以供日后使用后期發(fā)現(xiàn)問題也會(huì)繼續(xù)完善。參考文章https://blog.csdn.net/spiderwower/article/details/138755776但是我按照上述的例子只能做到一半所以又找其他一些相關(guān)的資料才完成所有任務(wù)。1、下載基模型使用魔搭社區(qū)https://modelscope.cn①首先先在魔搭社區(qū)找到需要使用的基模型比如我這里使用阿里的Qwen2.5-0.5模型完整名字Qwen/Qwen2.5-0.5B-Instruct②確保已安裝合適的conda環(huán)境和python環(huán)境這里使用python3.10具體不再細(xì)說。conda create-nnlppython3.10conda activate nlp310 切換到nlp310環(huán)境 conda activate nlp310 設(shè)置鏡像 pip configsetglobal.index-url https://pypi.tuna.tsinghua.edu.cn/simple③安裝torch環(huán)境如果有英偉達(dá)的顯卡一定要安裝gpu版本我這里筆記本性能有限MX3502G顯存沒辦法只能選擇小模型當(dāng)然要提前安裝好cuda環(huán)境這里計(jì)劃使用cuda版本為11.8。可以參考文章win114060配置CUDA11.8pytorch2.0.0好之后可以使用阿里云的鏡像源來安裝pip3installtorch2.6.0 torchvision torchaudio-fhttps://mirrors.aliyun.com/pytorch-wheels/cu118/#南京大學(xué)鏡像站pip3installtorch2.6.0 torchvision torchaudio-fhttps://mirrors.nju.edu.cn/pytorch/whl/cu118/可以通過如下地址驗(yàn)證版本及cuda是否可用python-cimport torch;print(torch.cuda.is_available())當(dāng)然也可以安裝其他版本。安裝完后再接著安裝訓(xùn)練微調(diào)使用的pip包transformers4.49.0streamlit1.24.0sentencepiece0.1.99accelerate0.29.3datasets2.14.5peft0.10.0tiktoken0.9.0 modelscope[framework]也有可能會(huì)報(bào)錯(cuò)原因是transformers的版本問題。這種情況下會(huì)建議安裝tf-keras。pipinstalltf-keras④基模型下載使用如下的代碼當(dāng)然要提前安裝modelscope環(huán)境變量中提前設(shè)置modelscope的默認(rèn)存儲(chǔ)地址我把下載路徑換到了d:/cuda/modelscope中。環(huán)境變量名MODELSCOPE_CACHE 環(huán)境值d:/cuda/modelscope模型下載md_download1.pyfrom modelscopeimportsnapshot_download#模型存放路徑# model_path /root/autodl-tmp#模型名字nameQwen/Qwen2.5-0.5B-Instruct# model_dir snapshot_download(name, cache_dirmodel_path, revisionmaster)model_dirsnapshot_download(name)注意name為你想下載的模型全名運(yùn)行即可下載python md_download1.py具體內(nèi)容2、下載訓(xùn)練集以中文數(shù)據(jù)集弱智吧為例約1500條對(duì)話數(shù)據(jù)來進(jìn)行訓(xùn)練,完整名稱為kigner/ruozhiba-llama3-tt地址為https://huggingface.co/datasets/kigner/ruozhiba-llama3-tt/tree/main我是手工下載文件到路徑d:/cuda/ruozhiba-llma3-tt。3、使用lora微調(diào)。①安裝相應(yīng)的python包。使用其他文章中的訓(xùn)練代碼只調(diào)整了模型名稱,train.pyimportos os.environ[TF_ENABLE_ONEDNN_OPTS]0fromdatasetsimportDatasetimportpandasaspdfromtransformersimport(AutoTokenizer,AutoModelForCausalLM,DataCollatorForSeq2Seq,TrainingArguments,Trainer,)importtorchfrompeftimportLoraConfig,TaskType,get_peft_modelimportwarnings warnings.filterwarnings(ignore,categoryUserWarning)# 忽略告警devicecudaiftorch.cuda.is_available()elsecpuprint(device:device)print(fPyTorch版本:{torch.__version__})print(f是否有可用GPU:{torch.cuda.is_available()})iftorch.cuda.is_available():print(fGPU數(shù)量:{torch.cuda.device_count()})print(f當(dāng)前GPU:{torch.cuda.current_device()})print(fGPU名稱:{torch.cuda.get_device_name(0)})# 模型文件路徑model_pathrD:/cuda/modelscope/models/Qwen/Qwen2___5-0___5B-Instruct# 訓(xùn)練過程數(shù)據(jù)保存路徑nameruozhibaoutput_dirf./output/qwen-0.5B-{name}#是否從上次斷點(diǎn)處接著訓(xùn)練如果需要從上次斷點(diǎn)處繼續(xù)訓(xùn)練值應(yīng)為Truetrain_with_checkpointFalse# 加載tokenizertokenizerAutoTokenizer.from_pretrained(model_path,use_fastFalse,trust_remote_codeTrue)tokenizer.pad_tokentokenizer.eos_token# 加載數(shù)據(jù)集dfpd.read_json(fD:/cuda/ruozhiba-llama3-tt/ruozhiba_qa.json)dsDataset.from_pandas(df)print(ds)# 對(duì)數(shù)據(jù)集進(jìn)行處理需要將數(shù)據(jù)集的內(nèi)容按大模型的對(duì)話格式進(jìn)行處理# 大模型處理defprocess_func_mistral(example):MAX_LENGTH384# Llama分詞器會(huì)將一個(gè)中文字切分為多個(gè)token因此需要放開一些最大長(zhǎng)度保證數(shù)據(jù)的完整性instructiontokenizer(fs[INST] SYS\n\n/SYS\n\n{example[instruction]example[input]}[/INST],add_special_tokensFalse)# add_special_tokens 不在開頭加 special_tokensresponsetokenizer(f{example[output]},add_special_tokensFalse)input_idsinstruction[input_ids]response[input_ids][tokenizer.pad_token_id]attention_maskinstruction[attention_mask]response[attention_mask][1]# 因?yàn)閜ad_token_id咱們也是要關(guān)注的所以 補(bǔ)充為1labels[-100]*len(instruction[input_ids])response[input_ids][tokenizer.pad_token_id]iflen(input_ids)MAX_LENGTH:# 做一個(gè)截?cái)鄆nput_idsinput_ids[:MAX_LENGTH]attention_maskattention_mask[:MAX_LENGTH]labelslabels[:MAX_LENGTH]return{input_ids:input_ids,attention_mask:attention_mask,labels:labels}inputs_idds.map(process_func_mistral,remove_columnsds.column_names)#加載模型# 根據(jù)GPU能力選擇合適的數(shù)據(jù)類型iftorch.cuda.is_available()andtorch.cuda.get_device_capability()[0]8:# A100、H100、RTX 40系列等支持bfloat16torch_dtypetorch.bfloat16else:# 其他GPU使用float16torch_dtypetorch.float16print(f使用的數(shù)據(jù)類型:{torch_dtype})ifnotos.path.exists(model_path):raiseValueError(f模型路徑不存在:{model_path})# 列出路徑下的文件檢查是否包含配置文件print(f模型路徑下的文件:{os.listdir(model_path)})modelAutoModelForCausalLM.from_pretrained(model_path,device_mapdevice,torch_dtypetorch.bfloat16,use_cacheFalse)model.enable_input_require_grads()# 開啟梯度檢查點(diǎn)時(shí)要執(zhí)行該方法print(model)#設(shè)置lora訓(xùn)練參數(shù)configLoraConfig(task_typeTaskType.CAUSAL_LM,target_modules[q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj],inference_modeFalse,# 訓(xùn)練模式r8,# Lora 秩lora_alpha32,# Lora alaph具體作用參見 Lora 原理lora_dropout0.1# Dropout 比例)#設(shè)置訓(xùn)練參數(shù)modelget_peft_model(model,config)model.print_trainable_parameters()argsTrainingArguments(output_diroutput_dir,# 模型保存路徑per_device_train_batch_size2,# 訓(xùn)練的批大小 之前為2顯存2G可用實(shí)際顯存最大會(huì)占到4.6G 32時(shí)顯存占用到15G16時(shí)顯存占用到15G8時(shí)顯存占用到11.7G4時(shí)顯存占用到6.7Ggradient_accumulation_steps2,# 梯度累積步數(shù)logging_steps10,# 日志打印的步數(shù)num_train_epochs2,# 訓(xùn)練的輪數(shù) 之前為2save_steps25,# 保存的步數(shù)save_total_limit2,# 保存的模型數(shù)量learning_rate1e-4,# 學(xué)習(xí)率save_on_each_nodeTrue,# 多節(jié)點(diǎn)訓(xùn)練時(shí)每個(gè)節(jié)點(diǎn)保存一次gradient_checkpointingTrue,# 開啟梯度檢查點(diǎn)label_names[labels],# 標(biāo)簽名稱)trainerTrainer(modelmodel,argsargs,train_datasetinputs_id,data_collatorDataCollatorForSeq2Seq(tokenizertokenizer,paddingTrue),)#開始訓(xùn)練# 如果訓(xùn)練中斷了還可以從上次中斷保存的位置繼續(xù)開始訓(xùn)練iftrain_with_checkpoint:checkpoint[fileforfileinos.listdir(output_dir)ifcheckpointinfile][-1]last_checkpointf{output_dir}/{checkpoint}print(last_checkpoint)trainer.train(resume_from_checkpointlast_checkpoint)else:trainer.train()執(zhí)行代碼python train.py會(huì)生成幾個(gè)checkpoint這個(gè)過程在我的電腦上2輪批次大小2個(gè)使用了差不多50分鐘精度也很差。0.5b訓(xùn)練截圖后來找了資源又試了其他的批次8,100輪訓(xùn)練A4000顯卡16G顯存loss降到了0.005情況{loss:0.005,grad_norm:0.1689453125,learning_rate:0.0,epoch:98.94}{train_runtime:4730.9062,train_samples_per_second:31.622,train_steps_per_second:1.966,train_loss:0.09664655848256042,epoch:98.94}100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|9300/9300[1:18:5000:00,1.97it/s]我的MX3502G顯存調(diào)1.5b模型的截圖②、checkpoint轉(zhuǎn)為loracheckpoint_to_lora.py具體代碼from transformersimportAutoModelForSequenceClassification,AutoTokenizerimportos# 需要保存的lora路徑lora_pathd:/cuda/lora/qwen-0__5B-lora-ruozhiba# 模型路徑model_pathD:/cuda/modelscope/models/Qwen/Qwen2___5-0___5B-Instruct# 檢查點(diǎn)路徑checkpoint_diroutput/qwen-0.5B-ruozhibacheckpoint[fileforfileinos.listdir(checkpoint_dir)ifcheckpoint-infile][-1]#選擇更新日期最新的檢查點(diǎn)modelAutoModelForSequenceClassification.from_pretrained(foutput/qwen-0.5B-ruozhiba/{checkpoint})# 保存模型model.save_pretrained(lora_path)tokenizerAutoTokenizer.from_pretrained(model_path,use_fastFalse,trust_remote_codeTrue)tokenizer.pad_tokentokenizer.eos_token# 保存tokenizertokenizer.save_pretrained(lora_path)執(zhí)行代碼python checkpoint_to_lora.py最終會(huì)把模型生成到d:/cuda/lora/qwen-0__5B-lora-ruozhiba中③、模型合并merge.py代碼如下from transformersimportAutoModelForCausalLM, AutoTokenizerimporttorch from peftimportPeftModel from peftimportLoraConfig, TaskType, get_peft_model model_pathD:/cuda/modelscope/models/Qwen/Qwen2___5-0___5B-Instructlora_pathd:/cuda/lora/qwen-0__5B-lora-ruozhibadevicecudaiftorch.cuda.is_available()elsecpu# 合并后的模型路徑output_pathrd:/cuda/merge# 等于訓(xùn)練時(shí)的config參數(shù)configLoraConfig(task_typeTaskType.CAUSAL_LM,target_modules[q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj],inference_modeFalse,# 訓(xùn)練模式r8,# Lora 秩lora_alpha32,# Lora alaph具體作用參見 Lora 原理lora_dropout0.1# Dropout 比例)baseAutoModelForCausalLM.from_pretrained(model_path,torch_dtypetorch.bfloat16,low_cpu_mem_usageTrue)base_tokenizerAutoTokenizer.from_pretrained(model_path)lora_modelPeftModel.from_pretrained(base, lora_path,torch_dtypetorch.float16,configconfig)modellora_model.merge_and_unload()model.save_pretrained(output_path)base_tokenizer.save_pretrained(output_path)執(zhí)行代碼python merge.py生成結(jié)果如下④模型轉(zhuǎn)為gguf格式下載源碼https://github.com/ggml-org/llama.cpp使用其中的轉(zhuǎn)換器轉(zhuǎn)換其實(shí)這里已經(jīng)轉(zhuǎn)為gguf格式只是是f16格式的第一種python convert_hf_to_gguf.py D:/cuda/merge--outtypef16--outfiled:/cuda/qwen-ruozhiba.bin也可以用如下指令量化為Q8的模型python convert_hf_to_gguf.py /myweb/cuda/merge--outtypeq8_0 此處處理過后會(huì)生成類似 Merge-0.5B-Q8_0.gguf 的文件4、模型量化注意如果僅用f16或者q8則如下可以不用再執(zhí)行。首先下載llama.cpp的編譯包https://github.com/ggml-org/llama.cpp/releases根據(jù)自己的cpu情況選擇合適的包比我的使用的cpu是intel10代處理通過cpuz可以查看支持的技術(shù)我這里支持AVX和AVX2可以使用AVX2的包下載后解壓會(huì)得到相關(guān)的編譯包可以將路徑加到path中但要注意避免沖突PS:后期升級(jí)ollama后出現(xiàn)了沖突去掉path即可。然后在D:/cuda中執(zhí)行轉(zhuǎn)換llama-quantize qwen-ruozhiba.bin q5_k_m會(huì)生成一個(gè)新文件ggml-model-Q5_K_M.gguf將文件放到新的目錄D:/cuda/qwen2.5-new5、安裝ollama。下載https://ollama.com/download/OllamaSetup.exe安裝。然后到文件夾中新增一個(gè)文件文件名ModelFile內(nèi)容如下第一種FROM D:/cuda/qwen2.5-new/ggml-model-Q5_K_M.gguf#FROM ./Merge-1.5B-F16.gguf#FROM D:/cuda/qwen2.5-new/ggml-model-Q5_K_M.gguf#FROM D:/cuda/qwen2.5-new/ggml-model-0.5_Q5_K_M.gguf# set the temperature to 0.7 [higher is more creative, lower is more coherent]PARAMETER temperature0.7PARAMETER top_p0.8PARAMETER repeat_penalty1.05TEMPLATE{{if.System}}|im_start|system{{.System}}|im_end|{{end}}{{if.Prompt}}|im_start|user{{.Prompt}}|im_end|{{end}}|im_start|assistant{{.Response}}|im_end|# set the system messageSYSTEM You are a helpful assistant.第二種FROM ./Merge-1.5B-F16.gguf# 修復(fù)1簡(jiǎn)化系統(tǒng)提示移除冗余條件判斷SYSTEM|im_start|system You are Qwen, created by Alibaba Cloud. You are a helpful assistant.|im_end|# 修復(fù)2完全重寫模板結(jié)構(gòu)TEMPLATE{{if.System}}|im_start|system{{.System}}|im_end|{{else}}|im_start|system You are Qwen, created by Alibaba Cloud. You are a helpful assistant.|im_end|{{end}}{{range .Messages}}{{ifeq .Roleuser}}|im_start|user{{.Content}}|im_end|{{elseifeq .Roleassistant}}|im_start|assistant{{.Content}}|im_end|{{elseifeq .Rolesystem}}|im_start|system{{.Content}}|im_end|{{end}}{{end}}|im_start|assistant第三種如果需要支持tool可以使用如下內(nèi)容#FROM qwen2.5:1.5bFROM ./Merge-1.5B-F16.gguf# set the temperature to 0.7 [higher is more creative, lower is more coherent]PARAMETER temperature0.7PARAMETER top_p0.8PARAMETER repeat_penalty1.05TEMPLATE{{-if.Messages}}{{-ifor .System .Tools}}|im_start|system{{-if.System}}{{.System}}{{- end}}{{-if.Tools}}# ToolsYou may call one ormorefunctions to assist with the user query. You are provided withfunctionsignatures withintools/toolsXML tags:tools{{- range .Tools}}{type:function,function:{{.Function}}}{{- end}}/toolsFor eachfunctioncall,returna json object withfunctionname and arguments withintool_call/tool_callXML tags:tool_call{name:function-name,arguments:args-json-object}/tool_call{{- end}}|im_end|{{end}}{{- range$i,$_:.Messages}}{{-$last:eq(len(slice $.Messages$i))1-}}{{-ifeq .Roleuser}}|im_start|user{{.Content}}|im_end|{{elseifeq .Roleassistant}}|im_start|assistant{{if.Content}}{{.Content}}{{-elseif.ToolCalls}}tool_call{{range .ToolCalls}}{name:{{ .Function.Name }},arguments:{{.Function.Arguments}}}{{end}}/tool_call{{- end}}{{ifnot$last}}|im_end|{{end}}{{-elseifeq .Roletool}}|im_start|usertool_response{{.Content}}/tool_response|im_end|{{end}}{{-ifand(ne .Roleassistant)$last}}|im_start|assistant{{end}}{{- end}}{{-else}}{{-if.System}}|im_start|system{{.System}}|im_end|{{end}}{{if.Prompt}}|im_start|user{{.Prompt}}|im_end|{{end}}|im_start|assistant{{end}}{{.Response}}{{if.Response}}|im_end|{{end}} SYSTEMYou are a helpful assistant, created by Alibaba Cloud and fine-tunedforspecific tasks. You have access to the following tools: - When appropriate, you can use tools to assist with the users query. - Tools will be provided in tools/tools XML tags. - For each function call, return a json object with function name and arguments within tool_call/tool_call XML tags. You should only use tools when necessary and beneficial to the users query.執(zhí)行創(chuàng)建ollama create qwen2_0.5b_demo--file./ModelFile最后運(yùn)行模型ollama run qwen2_0.5b_demo可以愉快的玩耍啦?？梢允褂胦llama show qwen2_0.5b_demo類似的指令來查看部署的模型類型。一般模型我今天又調(diào)了1.5b做測(cè)試內(nèi)容如下已經(jīng)支持tools了但是因?yàn)榇竽Ｐ蛯?shí)在太小沒什么效果后來訓(xùn)練的新模型還有些用有空也可以嘗試比較大的模型作為基模型來進(jìn)行訓(xùn)練。如果需要聯(lián)網(wǎng)思路可以參考文章https://blog.csdn.net/ChinaLiaoTian/article/details/145504774如果需要知識(shí)庫(kù)能力可以參考https://blog.csdn.net/sheex2012/article/details/138339166