{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Access token is needed for reading this data (contact Lavita's admin: pedram@lavita.ai)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reading data from Lavita's Hugging Face\n",
    "\n",
    "# data_hf_name = \"lavita/ChatDoctor-HealthCareMagic-100k\"\n",
    "data_hf_name = \"lavita/ChatDoctor-iCliniq\"\n",
    "dataset = load_dataset(data_hf_name)\n",
    "\n",
    "# Converting data to pandas DataFrame\n",
    "df = dataset['train'].to_pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input</th>\n",
       "      <th>answer_icliniq</th>\n",
       "      <th>answer_chatgpt</th>\n",
       "      <th>answer_chatdoctor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hello doctor,I had mumps five months ago and a...</td>\n",
       "      <td>Hello, Welcome to Chat Doctor forum. I can und...</td>\n",
       "      <td>It is possible that you still have an infectio...</td>\n",
       "      <td>1) There is no need to worry about reinfection...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hello doctor,I am trying to conceive but my hu...</td>\n",
       "      <td>Hello, Wellcome to ChatDoctor. There are few r...</td>\n",
       "      <td>Hello! Thank you for reaching out.\\n\\nCocaine ...</td>\n",
       "      <td>1) Cocaine use can affect fertility by causing...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Hello doctor,I am currently on the Yaz contrac...</td>\n",
       "      <td>Hello. The brown discharge should be temporary...</td>\n",
       "      <td>Hello! It's not uncommon to experience some ir...</td>\n",
       "      <td>1) The brown discharge is due to the hormonal ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Hello doctor,My son aged 13 years has got his ...</td>\n",
       "      <td>Hello, Welcome to Chat Doctor forum. I have go...</td>\n",
       "      <td>As an AI language model, I am not a doctor and...</td>\n",
       "      <td>Hi, welcome to Chat Doctor forum. I understand...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hello doctor,I just got one side of my wisdom ...</td>\n",
       "      <td>Hello. The lump is mostly a hard swelling whic...</td>\n",
       "      <td>Hello,\\n\\nIt is normal to have some swelling a...</td>\n",
       "      <td>Hi, Thanks for your query. It appears that you...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Hello doctor,I had deep unprotected sex for th...</td>\n",
       "      <td>Hi. Firstly, I want to tell you that the tests...</td>\n",
       "      <td>Hello,\\n\\nI understand your concern about poss...</td>\n",
       "      <td>Hi, welcome to Chat Doctor forum. As your sex ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Hello doctor,I need advice after my second CT ...</td>\n",
       "      <td>Hi. According to guidelines when there is a si...</td>\n",
       "      <td>Hello,\\n\\nBased on your CT results, it appears...</td>\n",
       "      <td>1) The nodules are probably benign and unlikel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hi doctor,I had sex for the first time. After ...</td>\n",
       "      <td>Hello. As the serum beta hCG levels are less t...</td>\n",
       "      <td>There are many possible reasons for late perio...</td>\n",
       "      <td>1. The delay in periods is usually due to stre...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hello doctor,My friend aged 30 had two drops o...</td>\n",
       "      <td>Hi. I want to assure you not to worry as every...</td>\n",
       "      <td>I'm sorry to hear about your friend's situatio...</td>\n",
       "      <td>1. The side effects of phenol include irritati...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Hello doctor,Some mornings during the week I g...</td>\n",
       "      <td>Hello.  I understand your concern and would ex...</td>\n",
       "      <td>Hello! It's difficult to give a definitive dia...</td>\n",
       "      <td>Hi, Welcome to Chat Doctor forum. The symptoms...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               input  \\\n",
       "0  Hello doctor,I had mumps five months ago and a...   \n",
       "1  Hello doctor,I am trying to conceive but my hu...   \n",
       "2  Hello doctor,I am currently on the Yaz contrac...   \n",
       "3  Hello doctor,My son aged 13 years has got his ...   \n",
       "4  Hello doctor,I just got one side of my wisdom ...   \n",
       "5  Hello doctor,I had deep unprotected sex for th...   \n",
       "6  Hello doctor,I need advice after my second CT ...   \n",
       "7  Hi doctor,I had sex for the first time. After ...   \n",
       "8  Hello doctor,My friend aged 30 had two drops o...   \n",
       "9  Hello doctor,Some mornings during the week I g...   \n",
       "\n",
       "                                      answer_icliniq  \\\n",
       "0  Hello, Welcome to Chat Doctor forum. I can und...   \n",
       "1  Hello, Wellcome to ChatDoctor. There are few r...   \n",
       "2  Hello. The brown discharge should be temporary...   \n",
       "3  Hello, Welcome to Chat Doctor forum. I have go...   \n",
       "4  Hello. The lump is mostly a hard swelling whic...   \n",
       "5  Hi. Firstly, I want to tell you that the tests...   \n",
       "6  Hi. According to guidelines when there is a si...   \n",
       "7  Hello. As the serum beta hCG levels are less t...   \n",
       "8  Hi. I want to assure you not to worry as every...   \n",
       "9  Hello.  I understand your concern and would ex...   \n",
       "\n",
       "                                      answer_chatgpt  \\\n",
       "0  It is possible that you still have an infectio...   \n",
       "1  Hello! Thank you for reaching out.\\n\\nCocaine ...   \n",
       "2  Hello! It's not uncommon to experience some ir...   \n",
       "3  As an AI language model, I am not a doctor and...   \n",
       "4  Hello,\\n\\nIt is normal to have some swelling a...   \n",
       "5  Hello,\\n\\nI understand your concern about poss...   \n",
       "6  Hello,\\n\\nBased on your CT results, it appears...   \n",
       "7  There are many possible reasons for late perio...   \n",
       "8  I'm sorry to hear about your friend's situatio...   \n",
       "9  Hello! It's difficult to give a definitive dia...   \n",
       "\n",
       "                                   answer_chatdoctor  \n",
       "0  1) There is no need to worry about reinfection...  \n",
       "1  1) Cocaine use can affect fertility by causing...  \n",
       "2  1) The brown discharge is due to the hormonal ...  \n",
       "3  Hi, welcome to Chat Doctor forum. I understand...  \n",
       "4  Hi, Thanks for your query. It appears that you...  \n",
       "5  Hi, welcome to Chat Doctor forum. As your sex ...  \n",
       "6  1) The nodules are probably benign and unlikel...  \n",
       "7  1. The delay in periods is usually due to stre...  \n",
       "8  1. The side effects of phenol include irritati...  \n",
       "9  Hi, Welcome to Chat Doctor forum. The symptoms...  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "qa-env",
   "language": "python",
   "name": "qa-env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}