model

README.md

@@ -19,7 +19,6 @@ tasks:
 <a href="https://github.com/baichuan-inc/Baichuan2" target="_blank">🦉GitHub</a> | <a href="https://github.com/baichuan-inc/Baichuan-7B/blob/main/media/wechat.jpeg?raw=true" target="_blank">💬WeChat</a>
 </div>
 <div align="center">
-  百川API支持搜索增强和192K长窗口，新增百川搜索增强知识库、限时免费！<br>
 🚀 <a href="https://www.baichuan-ai.com/" target="_blank">百川大模型在线对话平台</a> 已正式向公众开放 🎉
 </div>
 
@@ -28,13 +27,8 @@ tasks:
 - [📖 模型介绍/Introduction](#Introduction)
 - [⚙️ 快速开始/Quick Start](#Start)
 - [📊 Benchmark评估/Benchmark Evaluation](#Benchmark)
-- [👥 社区与生态/Community](#Community)
 - [📜 声明与协议/Terms and Conditions](#Terms)
 
-# 更新/Update
-[2023.12.29] 🎉🎉🎉 我们发布了 **[Baichuan2-13B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat) v2** 版本。其中：
-- 大幅提升了模型的综合能力，特别是数学和逻辑推理、复杂指令跟随能力。
-- 使用时需指定revision=v2.0，详细方法参考[快速开始](#Start)
 
 # <span id="Introduction">模型介绍/Introduction</span>
 
@@ -64,16 +58,9 @@ In the Baichuan 2 series models, we have utilized the new feature `F.scaled_dot_
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 from transformers.generation.utils import GenerationConfig
-tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/Baichuan2-13B-Chat",
-    revision="v2.0",
-    use_fast=False,
-    trust_remote_code=True)
-model = AutoModelForCausalLM.from_pretrained("baichuan-inc/Baichuan2-13B-Chat",
-    revision="v2.0",
-    device_map="auto",
-    torch_dtype=torch.bfloat16,
-    trust_remote_code=True)
-model.generation_config = GenerationConfig.from_pretrained("baichuan-inc/Baichuan2-13B-Chat", revision="v2.0")
+tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/Baichuan2-13B-Chat", use_fast=False, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained("baichuan-inc/Baichuan2-13B-Chat", device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True)
+model.generation_config = GenerationConfig.from_pretrained("baichuan-inc/Baichuan2-13B-Chat")
 messages = []
 messages.append({"role": "user", "content": "解释一下“温故而知新”"})
 response = model.chat(tokenizer, messages)
@@ -82,7 +69,6 @@ print(response)
 
 这句话鼓励我们在学习和生活中不断地回顾和反思过去的经验，从而获得新的启示和成长。通过重温旧的知识和经历，我们可以发现新的观点和理解，从而更好地应对不断变化的世界和挑战。
 ```
-**注意：如需使用老版本，需手动指定revision参数，设置revision=v1.0** 
 
 # <span id="Benchmark">Benchmark 结果/Benchmark Evaluation</span>
 
@@ -129,16 +115,6 @@ In addition to the [Baichuan2-7B-Base](https://huggingface.co/baichuan-inc/Baich
 
 ![checkpoint](https://huggingface.co/baichuan-inc/Baichuan2-7B-Base/resolve/main/checkpoints.jpeg)
 
-# <span id="Community">社区与生态/Community</span>
-
-## Intel 酷睿 Ultra 平台运行百川大模型
-
-使用酷睿™/至强® 可扩展处理器或配合锐炫™ GPU等进行部署[Baichuan2-7B-Chat]，[Baichuan2-13B-Chat]模型，推荐使用 BigDL-LLM([CPU], [GPU])以发挥更好推理性能。
-
-详细支持信息可参考[中文操作手册](https://github.com/intel-analytics/bigdl-llm-tutorial/tree/main/Chinese_Version)，包括用notebook支持，[加载，优化，保存方法](https://github.com/intel-analytics/bigdl-llm-tutorial/blob/main/Chinese_Version/ch_3_AppDev_Basic/3_BasicApp.ipynb)等。
-
-When deploy on Core™/Xeon® Scalable Processors or with Arc™ GPU, BigDL-LLM ([CPU], [GPU]) is recommended to take full advantage of better inference performance.
-
 # <span id="Terms">声明与协议/Terms and Conditions</span>
 
 ## 声明
@@ -156,21 +132,9 @@ We have done our best to ensure the compliance of the data used in the model tra
 
 ## 协议
 
-社区使用 Baichuan 2 模型需要遵循 [Apache 2.0](https://github.com/baichuan-inc/Baichuan2/blob/main/LICENSE) 和[《Baichuan 2 模型社区许可协议》](https://huggingface.co/baichuan-inc/Baichuan2-7B-Base/resolve/main/Baichuan%202%E6%A8%A1%E5%9E%8B%E7%A4%BE%E5%8C%BA%E8%AE%B8%E5%8F%AF%E5%8D%8F%E8%AE%AE.pdf)。Baichuan 2 模型支持商业用途，如果您计划将 Baichuan 2 模型或其衍生品用于商业目的，请您确认您的主体符合以下情况：
-  1. 您或您的关联方的服务或产品的日均用户活跃量（DAU）低于100万。
-  2. 您或您的关联方不是软件服务提供商、云服务提供商。
-  3. 您或您的关联方不存在将授予您的商用许可，未经百川许可二次授权给其他第三方的可能。
-
-在符合以上条件的前提下，您需要通过以下联系邮箱 [email protected] ，提交《Baichuan 2 模型社区许可协议》要求的申请材料。审核通过后，百川将特此授予您一个非排他性、全球性、不可转让、不可再许可、可撤销的商用版权许可。
-
-The community usage of Baichuan 2 model requires adherence to [Apache 2.0](https://github.com/baichuan-inc/Baichuan2/blob/main/LICENSE) and [Community License for Baichuan2 Model](https://huggingface.co/baichuan-inc/Baichuan2-7B-Base/resolve/main/Baichuan%202%E6%A8%A1%E5%9E%8B%E7%A4%BE%E5%8C%BA%E8%AE%B8%E5%8F%AF%E5%8D%8F%E8%AE%AE.pdf). The Baichuan 2 model supports commercial use. If you plan to use the Baichuan 2 model or its derivatives for commercial purposes, please ensure that your entity meets the following conditions:
-
-  1. The Daily Active Users (DAU) of your or your affiliate's service or product is less than 1 million.
-  2. Neither you nor your affiliates are software service providers or cloud service providers.
-  3. There is no possibility for you or your affiliates to grant the commercial license given to you, to reauthorize it to other third parties without Baichuan's permission.
-
-Upon meeting the above conditions, you need to submit the application materials required by the Baichuan 2 Model Community License Agreement via the following contact email: [email protected]. Once approved, Baichuan will hereby grant you a non-exclusive, global, non-transferable, non-sublicensable, revocable commercial copyright license.
+Baichuan 2 模型的社区使用需遵循[《Baichuan 2 模型社区许可协议》]。Baichuan 2 支持商用。如果将 Baichuan 2 模型或其衍生品用作商业用途，请您按照如下方式联系许可方，以进行登记并向许可方申请书面授权：联系邮箱 [[email protected]]。
 
+The use of the source code in this repository follows the open-source license Apache 2.0. Community use of the Baichuan 2 model must adhere to the [Community License for Baichuan 2 Model](https://huggingface.co/baichuan-inc/Baichuan2-7B-Base/blob/main/Baichuan%202%E6%A8%A1%E5%9E%8B%E7%A4%BE%E5%8C%BA%E8%AE%B8%E5%8F%AF%E5%8D%8F%E8%AE%AE.pdf). Baichuan 2 supports commercial use. If you are using the Baichuan 2 models or their derivatives for commercial purposes, please contact the licensor in the following manner for registration and to apply for written authorization: Email [email protected].
 
 [GitHub]:https://github.com/baichuan-inc/Baichuan2
 [Baichuan2]:https://github.com/baichuan-inc/Baichuan2
@@ -198,6 +162,3 @@ Upon meeting the above conditions, you need to submit the application materials
 [[email protected]]: mailto:[email protected]
 [训练过程heckpoint下载]: https://huggingface.co/baichuan-inc/Baichuan2-7B-Intermediate-Checkpoints
 [百川智能]: https://www.baichuan-ai.com
-
-[CPU]: https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/HF-Transformers-AutoModels/Model/baichuan2
-[GPU]: https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/GPU/HF-Transformers-AutoModels/Model/baichuan2

configuration_baichuan.py

@@ -9,7 +9,7 @@ class BaichuanConfig(PretrainedConfig):
 
     def __init__(
         self,
-        vocab_size=125696,
+        vocab_size=64000,
         hidden_size=5120,
         intermediate_size=13696,
         num_hidden_layers=40,

handler.py

@@ -1,23 +0,0 @@
-import torch
-from typing import Dict, List, Any
-from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
-from transformers.generation.utils import GenerationConfig
-
-# get dtype
-dtype = torch.bfloat16 if torch.cuda.get_device_capability()[0] == 8 else torch.float16
-
-class EndpointHandler:
-    def __init__(self, path=""):
-        # load the model
-        self.model = AutoModelForCausalLM.from_pretrained(path, device_map="auto", torch_dtype=dtype, trust_remote_code=True)
-        self.model.generation_config = GenerationConfig.from_pretrained(path)
-        self.tokenizer = AutoTokenizer.from_pretrained(path, use_fast=False, trust_remote_code=True)
-
-    def __call__(self, data: Any) -> List[List[Dict[str, float]]]:
-        inputs = data.pop("inputs", data)
-        # ignoring parameters! Default to configs in generation_config.json.
-        messages = [{"role": "user", "content": inputs}]
-        response = self.model.chat(self.tokenizer, messages)
-        if torch.backends.mps.is_available():
-            torch.mps.empty_cache()
-        return [{'generated_text': response}]

tokenization_baichuan.py

@@ -68,13 +68,6 @@ class BaichuanTokenizer(PreTrainedTokenizer):
             if isinstance(pad_token, str)
             else pad_token
         )
-
-        self.vocab_file = vocab_file
-        self.add_bos_token = add_bos_token
-        self.add_eos_token = add_eos_token
-        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
-        self.sp_model.Load(vocab_file)
-        
         super().__init__(
             bos_token=bos_token,
             eos_token=eos_token,
@@ -86,6 +79,11 @@ class BaichuanTokenizer(PreTrainedTokenizer):
             clean_up_tokenization_spaces=clean_up_tokenization_spaces,
             **kwargs,
         )
+        self.vocab_file = vocab_file
+        self.add_bos_token = add_bos_token
+        self.add_eos_token = add_eos_token
+        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
+        self.sp_model.Load(vocab_file)
 
     def __getstate__(self):
         state = self.__dict__.copy()