自从2022年底，OpenAI推出了ChatGPT，许多人认为这是AI的iPhone时刻。然而，OpenAI的聊天机器人并不是第一个生成式AI文本机器学习模型，它是在两年前推出的GPT-3之后推出的。

OpenAI为我们提供了一个已经训练好的GPT-3模型，此外，可以在较小的数据集上对特定任务进行微调。例如，如果你想创建一个针对你公司特定的电子邮件响应生成器，首先必须收集大量关于你特定业务领域的数据，例如客户电子邮件查询和响应。然后，你可以使用这些数据对GPT-3进行微调，使其学习你公司的特定语言模式和短语。通过微调GPT-3，可以创建一个高度定制和专业化的电子邮件响应生成器，专门针对特定业务领域中使用的语言模式和词汇。

在本博客文章中，我将向你展示如何微调GPT-3。我们将使用Python代码完成，并不需要对GPT-3有任何先前的了解。

微调所需的内容

与在Hugging Face上可用的GPT-2模型不同（当撰写此博客时），我们没有直接访问GPT-3模型。因此，你需要先从OpenAI获取API密钥并安装Python包_openai，可以通过pip快速完成。

对于OpenAI的API密钥：

• 前往https://platform.openai.com/account/api-keys，
• 创建一个帐户，
• 点击“Create new secret key”，然后
• 复制密钥。

该密钥是以“sk-”开头的一长串字符。确保你保密！一旦获得了密钥，获取密钥的简单方法是在终端中执行以下操作：（出于简单起见，我将其放在了我的_.bachrc_中）：

export OPENAI_API_KEY=sk-t59pgejhtrff5(...)

使用GPT-3模型需要成本。我们需要积分。在撰写本文时，当你创建一个新帐户时，你会获得免费积分以尝试这个工具。我不知道这是否会继续...

现在，我们拥有了我们的密钥和Python包，是时候考虑我们需要微调的数据了。首先，我们需要一个用于微调的示例文件，其中每个示例都是_prompt_后跟适当的_completion_。

超级英雄描述生成工具

我们将构建一个工具来创建虚构超级英雄的描述。最终，该工具将接收超级英雄的年龄、性别和力量，并自动产生我们的超级英雄的描述。

在下面的示例中，在微调模型之后，我们只需要说“40，女性，治愈 ->”，我们就可以从模型中自动获得一个描述。

这就是这一切的意义所在！😃

创建微调合成数据集

在某些情况下，你可能有一个数据集，你想要用于微调。但由于我没有一个，让我们看看如何直接从GPT-3中创建具有超级英雄描述的合成数据集。以下代码将为我提供一个包含_prompts_和相应_completion_的CSV文件。

import os
import openai
import pandas as pd

openai.api_key = os.getenv("OPENAI_API_KEY")

l_age = ['18', '20', '30', '40', '50', '60', '90']
l_gender = ['man', 'woman']
l_power = ['invisibility', 'read in the thoughts', 'turning lead into gold', 'immortality', 'telepathy', 'teleport', 'flight'] 

f_prompt = "Imagine a complete and detailed description of a {age}-year-old {gender} fictional character who has the superpower of {power}. Write out the entire description in a maximum of 100 words in great detail:"
f_sub_prompt = "{age}, {gender}, {power}"

df = pd.DataFrame()
for age in l_age:
 for gender in l_gender:
  for power in l_power:
   for i in range(3): ## 3 times each
    prompt = f_prompt.format(age=age, gender=gender, power=power)
    sub_prompt = f_sub_prompt.format(age=age, gender=gender, power=power)
    print(sub_prompt)

    response = openai.Completion.create(
     model="text-davinci-003",
     prompt=prompt,
     temperature=1,
     max_tokens=500,
     top_p=1,
     frequency_penalty=0,
     presence_penalty=0
    )
    
    finish_reason = response['choices'][0]['finish_reason']
    response_txt = response['choices'][0]['text']
    
    new_row = {
      'age':age, 
      'gender':gender, 
      'power':power, 
      'prompt':prompt, 
      'sub_prompt':sub_prompt, 
      'response_txt':response_txt, 
      'finish_reason':finish_reason}
    new_row = pd.DataFrame([new_row])
    df = pd.concat([df, new_row], axis=0, ignore_index=True)

df.to_csv("out_openai_completion.csv")

让我们看看这段代码是如何工作的🧐。

变量_f_prompt_包含以下句子，其中缺少_{age}_, {gender}和{power}。

想象一下一个完整而详细的{age}岁的{gender}虚构角色的描述，他拥有{power}的超能力。在100个单词的最大限度内详细描述整个描述:

在代码的前三个_for_循环中，我们迭代不同的{age}、{gender}和{power}的值。在循环的每一步中，我们使用不同的值替换三个缺失的变量。然后，我们使用_openai.Completion.create_函数要求GPT根据我们的提示生成响应。

该函数的最重要的参数是：

• model：用于生成响应的模型。OpenAI提供四种标准的GPT-3模型（ada、babbage、curie或davinci），它们的大小不同，使用价格也不同。这里是_davinci_——最大的模型。
• prompt：我们想要用GPT-3来完成的提示。
• temperature：温度是0到1之间的数字，控制输出中有多少随机性。我们将温度设置为最大值，以允许模型在创建响应时尽可能具有创造力。
• max_tokens：定义响应的最大长度。

在此脚本的末尾，我们有一个存储在文件_out_openai_completion.csv_中的Pandas表。在这个表中，我们感兴趣的两个主要列是_sub_prompt_和_response_txt_。

• _sub_prompt_将是例如'_18，男性，隐形'。它包含三个被替换的值，由逗号分隔。
• _response_txt_包含GPT模型的输出。

微调GPT模型

以下代码检索先前创建的文件_out_openai_completion.csv，并使用_openai_来微调GPT-3模型。

import pandas as pd
import openai
import subprocess

df = pd.read_csv("out_openai_completion.csv")

prepared_data = df.loc[:,['sub_prompt','response_txt']]
prepared_data.rename(columns={'sub_prompt':'prompt', 'response_txt':'completion'}, inplace=True)
prepared_data.to_csv('prepared_data.csv',index=False)

## prepared_data.csv --> prepared_data_prepared.json
subprocess.run('openai tools fine_tunes.prepare_data --file prepared_data.csv --quiet'.split())

## Start fine-tuning
subprocess.run('openai api fine_tunes.create --training_file prepared_data_prepared.jsonl --model davinci --suffix "SuperHero"'.split())

让我们花点时间理解这段代码🤓！首先，将文件_out_openai_completion.csv_中的内容加载到数据框_df_中。作为提醒，在我们的任务中，如果用户输入“40岁、女性、治愈”，我们希望有一个40岁女性角色的描述，具有治愈能力。为了进行精细调整，需要向GPT提供用户可能输入的示例和相应的期望响应。在数据框_df_中，列_sub_prompt_和_response_txt_包含输入示例和相应的期望响应。在上面的代码中，我们首先提取这两个冒号，然后将它们分别重命名为_prompt_和_completion_。生成的数据框存储在一个名为_prepared_data.csv_的新文件中，该文件仅包含这两列。

这个文件_prepared_data.csv_长这样：

prompt,completion
"18, man, invisibility","He looks to be in his late teens, with dark eyes and unruly black hair which reach his shoulders. His build is athletic and strong, but not muscular. He often wears clothing that makes him blend in with his surroundings, in order to stay as inconspicuous as possible.He has the ability of invisibility, which he can use to become an invisible observer or to make himself or objects around him disappear from view. He has honed his skills to make himself undetectable, able to even make sounds vanish, allowing him to go unnoticed. His invisibility power is his greatest asset, and he uses it to protect those he cares about."
"18, man, invisibility","18 year-old Jackson is a powerful young man with the incredible power of invisibility. He stands at 6'2 and has a strong athletic frame with wavy brown hair, startling blue eyes and a warm and gentle smile. His strength and agility make him especially good at using his ability. He can become invisible in an instant, moving through walls and other obstacles without being detected. What's more, he can move objects with his mind and use his power to protect those he loves. His power is a blessing and a curse, as it can be abused to do harm. Jackson is a brave and noble person who uses his power to do good and make the world a better place."
"18, man, invisibility","Brandon is an 18-year-old of average build, standing roughly 5'10 with an inquisitive look. He has naturally wavy chestnut brown hair and bright blue eyes. His demeanor is usually quite serious, but he also has an easy and gentle smile. He has a natural gift of invisibility, which he uses to protect himself and his family from harm. He's an inquisitive and creative young man who uses his gift to explore the world, observe people, and uncover the truth. His consistent practice of mindfulness helps him remain unseen, even when his emotions are visible. His intelligence and persistent passion for truth drives him to help those in need."
"18, man, read in the thoughts","This 18-year-old man has a muscular stature and long, ash blonde hair. His bright blue eyes are full of knowledge and power, hinting at the strange ability he has - he can read minds. Now and then, strange sparks of electricity streak from his eyes when he concentrates hard enough. He is often calm and collected, but when provoked has the ability to blend his learning of your thoughts with his immense strength. His wit, intelligence and empathy are weapons that are often underestimated - yet, very lethal."
"18, man, read in the thoughts","He's an 18-year-old man with short, cropped dark hair and bright green eyes. His usual outfit consists of an olive green hoodie, blue jeans and purple trainers. He's relaxed, confident and always reads the atmosphere well. One of his great skills is read in the thoughts of others. By doing so, he is able to understand people more easily and relate to their concerns out of the blue. He always uses his superpower with the utmost dignity and respect, never using it to manipulate. He uses it to seek the truth, identify people's emotions and gain insight into their perspectives. He's the sort of person that can make anyone feel comfortable and is never afraid of a challenge. His superpower is merely an extension of his already charismatic disposition."
(...)

Python的_subprocess.run()_函数作为子进程运行命令。它经常用于执行外部程序，就像在终端中运行一样。

我们使用_subprocess.run()_来执行“openai tools fine_tunes.prepare_data”的命令。这个函数接受输入文件_prepared_data.csv，检查数据是否正确，并生成一个名为_prepared_data_prepared.jsonl_的JSONL文件。JSONL文件是一种格式，它将每个JSON对象存储在单独的行上。JSONL文件包含一系列JSON对象，每个对象都由一个换行符分隔。

请注意，我们添加了选项“- - quiet”以自动接受“openai tools fine_tunes.prepare_data”提出的所有建议。例如，它建议在所有提示的末尾添加一个“-->”并在每个响应的末尾添加一个令牌_END_。

这个JSONL文件的前几行看起来像这样：

{"prompt":"18, man, invisibility ->","completion":" \n\nHe looks to be in his late teens, with dark eyes and unruly black hair which reach his shoulders. His build is athletic and strong, but not muscular. He often wears clothing that makes him blend in with his surroundings, in order to stay as inconspicuous as possible.\n\nHe has the ability of invisibility, which he can use to become an invisible observer or to make himself or objects around him disappear from view. He has honed his skills to make himself undetectable, able to even make sounds vanish, allowing him to go unnoticed. His invisibility power is his greatest asset, and he uses it to protect those he cares about. END"}
{"prompt":"18, man, invisibility ->","completion":" \n\n18 year-old Jackson is a powerful young man with the incredible power of invisibility. He stands at 6'2 and has a strong athletic frame with wavy brown hair, startling blue eyes and a warm and gentle smile. His strength and agility make him especially good at using his ability. He can become invisible in an instant, moving through walls and other obstacles without being detected. What's more, he can move objects with his mind and use his power to protect those he loves. His power is a blessing and a curse, as it can be abused to do harm. Jackson is a brave and noble person who uses his power to do good and make the world a better place. END"}
(...)

GPT-3模型的微调实际上是在第二个_subprocess.run()中完成的，其中执行_openai api fine_tunes.create。在这个函数中，我们首先给出刚刚创建的JSONL文件的名称。然后，你需要选择要微调的模型。OpenAI提供了四个主要模型，具有不同的性能水平，适用于各种任务。_Davinci_是最强大的模型，Ada是最快的。_Davinci_也是最昂贵的模型😨。

由于我的模型的目的是创建超级英雄的描述，所以我们给我的新模型添加后缀“Superhero”。

就是这样😉几分钟后，你就会有一个可以使用的微调模型🌟。

现在是测试你的新模型的时候了

有不同的方法可以使用模型完成。主要是通过OpenAI提供的Playground或通过像Python这样的编程语言。

最简单的方法可能是使用playground。

• 前往https://platform.openai.com/playground。
• 点击“模型”并搜索带有后缀“Superhero”的模型。
• 在“停止序列”中添加令牌“END”。

现在是时候向我们的模型询问新预测了。我们将要求描述一个18岁的男性角色，他真的拥有一个不必要的能力😉我们将要求描述一个具有能力的角色...“可以吃很多东西”...看看会发生什么... 😆

不错😅

你想在Python中做吗？简单！点击屏幕右上角的“查看代码”。

在我们的情况下，“查看代码”中有这个：

import os
import openai

openai.api_key = os.getenv("OPENAI_API_KEY")

response = openai.Completion.create(
  model="davinci:ft-klhm:superhero-2023-02-01-14-56-48",
  prompt="18, Man, can eat a lot ->\n",
  temperature=0.7,
  max_tokens=256,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0,
  stop=["END"]
)

你只需要复制并粘贴它👍。

结论

在本博客中，我们已经看到如何生成合成数据来完善我们的模型以及如何进行微调。我们使用了创建超级英雄的用例，但是相同的方法可以用于你可能拥有的任何用例。最重要的是要有足够的质量示例，包括提示和期望响应。

译自：https://towardsdatascience.com/unleashing-the-power-of-gpt-how-to-fine-tune-your-model-da35c90766c4