下载huggingface

• huggingface 官网：https://huggingface.co/
• 国内镜像：https://hf-mirror.com/

1. url

获取 huggingface 上模型和数据文件的 url。

from huggingface_hub import hf_hub_url
from huggingface_hub.utils import filter_repo_objects
from huggingface_hub.hf_api import HfApi

repo_id = "decapoda-research/llama-7b-hf"
repo_type = "model" # 如果是数据 dataset

repo_info = HfApi().repo_info(repo_id=repo_id, repo_type=repo_type) # 有时候会连接Error，多试几次
files = list(filter_repo_objects(items=[f.rfilename for f in repo_info.siblings]))
urls = [hf_hub_url(repo_id, filename=file, repo_type=repo_type) for file in files]
print("\n".join(urls))

2. 镜像

详细下载方式参考：国内镜像.