Update README.md

README.md

@@ -3,197 +3,335 @@ library_name: transformers
 tags: []
 ---
 
-# Model Card for Model ID
+# How to use
 
-<!-- Provide a quick summary of what the model is/does. -->
+We have tested (and thus recommend) running this model on vLLM. We recommend running it from the vLLM openAI server, using the following command:
 
+```bash
+python -m vllm.entrypoints.openai.api_server --model lightblue/Mixtral-8x22B-v0.1-instruct --tensor-parallel-size 4 --gpu-memory-utilization 0.95 --max-model-len 1024
+```
+which is how we ran it on a 4 x A100 (80GB) machine.
 
+You can then call this model from Python installing the openai package:
 
-## Model Details
+```bash
+pip install openai
+```
 
-### Model Description
+and calling the model like so:
 
-<!-- Provide a longer summary of what this model is. -->
+```python
+from openai import OpenAI
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+vllm_client = OpenAI(
+    api_key="EMPTY",
+    base_url="http://localhost:8000/v1",
+)
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+prompt = "Tell me three cool facts about fish to amaze my 4 year old."
 
-### Model Sources [optional]
+response = vllm_client.chat.completions.create(
+    messages=[
+        {"role": "user", "content": prompt}, 
+         ],
+    model="/workspace/llm_training/axolotl/mixtral_8x22B_training/merged_model_multiling",
+    temperature=0,
+    max_tokens=900,
+)
 
-<!-- Provide the basic links for the model. -->
+print("### Prompt")
+print(prompt)
+print()
+print("### Response")
+print(response.choices[0].message.content)
+print()
+```
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+We will be uploading a 4bit AWQ model soon to make it easier to run this model on other machines (watch this space!).
 
-## Uses
+# Inference examples
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
-### Direct Use
+<details>
+  <summary>Creative prompts</summary>
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+```text
+### Prompt
+Write a hilarious 200 word story about Chimpanzees at the zoo
 
-[More Information Needed]
+### Response
+ Once upon a time in the bustling metropolis of Chimp City, there lived a group of mischievous chimpanzees who called the local zoo their home. These primates were known for their witty antics and their uncanny ability to make even the most serious of humans chuckle.
 
-### Downstream Use [optional]
+One sunny afternoon, the zookeeper, a stern fellow named Mr. Grumpkins, was making his rounds when he noticed something peculiar. The chimpanzees were huddled together, whispering and giggling amongst themselves. Curious, Mr. Grumpkins approached the enclosure, only to be met with a barrage of banana peels and monkey business.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+It turned out that the chimps had been secretly practicing their stand-up comedy routines, and they were ready to debut their act to the unsuspecting zoo visitors. The first chimp, named Chuckles, took the stage and began his routine. "Why did the banana go to the doctor? Because it wasn't peeling well!" The crowd erupted in laughter, and Mr. Grumpkins couldn't help but crack a smile.
 
-[More Information Needed]
+Next up was Giggles, who delivered a knock-knock joke that had the audience in stitches. "Knock, knock!" "Who's there?" "Banana." "Banana who?" "Banana split! Get it? Because I'm a chimp and I love bananas!" The crowd roared with laughter, and even Mr. Grumpkins found himself chuckling.
 
-### Out-of-Scope Use
+As the show continued, the chimps' comedic talents only grew, with each joke more hilarious than the last. The zoo visitors were thoroughly entertained, and even Mr. Grumpkins had to admit that the chimps had a knack for making people laugh.
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+In the end, the chimpanzees' stand-up comedy show became a hit, with visitors flocking to the zoo just to catch a glimpse of the hilarious primates. Mr. Grumpkins, now a reformed man, even joined in on the fun, sharing his own jokes with the chimps and the crowd. And so, the legend of the Chimp City Comedy Club was born, bringing laughter and joy to all who visited the zoo.
 
-[More Information Needed]
+```
 
-## Bias, Risks, and Limitations
+```text
+### Prompt
+Write 10 jokes that I can use at my boss' retirement party. Nothing too risque as that will get me fired.
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+### Response
+ 1. As we bid farewell to our dear boss, let's remember that retirement is like a never-ending vacation, except you don't have to worry about your boss calling you in the middle of it!
+2. Our boss is retiring, and we're all wondering what he'll do with all that free time. I hear he's planning to take up a new hobby: managing his own retirement!
+3. In honor of our boss's retirement, we've decided to give him a special gift: a lifetime supply of coffee, so he can finally get some rest!
+4. Our boss has been a great leader, always guiding us through the ups and downs of work life. Now, he's retiring to focus on the ups and downs of his golf game!
+5. As our boss embarks on his retirement journey, we'd like to remind him that he's not just leaving a job, he's leaving a legacy of hard work, dedication, and a whole lot of paperwork!
+6. Our boss is retiring, and we're all wondering what he'll do with his time. I hear he's planning to take up a new hobby: managing his own retirement!
+7. In honor of our boss's retirement, we've decided to give him a special gift: a lifetime supply of coffee, so he can finally get some rest!
+8. Our boss has been a great leader, always guiding us through the ups and downs of work life. Now, he's retiring to focus on the ups and downs of his golf game!
+9. As our boss embarks on his retirement journey, we'd like to remind him that he's not just leaving a job, he's leaving a legacy of hard work, dedication, and a whole lot of paperwork!
+10. Our boss is retiring, and we're all wondering what he'll do with his time. I hear he's planning to take up a new hobby: managing his own retirement!
+```
 
-[More Information Needed]
+</details>
 
-### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+<details>
+  <summary>Factual prompts</summary>
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+```text
+### Prompt
+Tell me about Strathaven, Scotland.
 
-## How to Get Started with the Model
+### Response
+ Strathaven is a charming town located in South Lanarkshire, Scotland. It is situated approximately 20 miles (32 kilometers) south of Glasgow and 15 miles (24 kilometers) east of East Kilbride. The town has a rich history and is known for its picturesque setting, historic buildings, and annual events.
 
-Use the code below to get started with the model.
+Strathaven's origins date back to the 12th century when it was established as a market town. The town's name comes from the Gaelic word "Strath" meaning "valley" and "Aven" meaning "river," referring to the River Avon that runs through the area.
 
-[More Information Needed]
+One of the most notable landmarks in Strathaven is the Strathaven Castle, also known as Avondale Castle. The castle dates back to the 14th century and was once an important stronghold for the Barons of Avondale. Although it is now in ruins, it remains a popular attraction for visitors who can explore its remains and learn about its history.
 
-## Training Details
+The town center features several historic buildings, including the Town Mill, which dates back to the 17th century and has been restored as a museum showcasing local history and crafts. The John Hastie Museum Trust also operates a small museum in Strathaven dedicated to preserving local heritage.
 
-### Training Data
+Strathaven hosts several annual events that draw visitors from near and far. One of the most famous events is the Strathaven Balloon Festival, which takes place every August and features hot air balloons from around the world taking off from John Hastie Park. Other events include the Strathaven Gala Day in June, which includes a parade, live music, and various stalls selling local produce and crafts.
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+The surrounding countryside offers opportunities for outdoor activities such as walking, cycling, fishing, and golfing at nearby courses like Strathaven Golf Club or Dalserf Golf Club. The nearby Clyde Valley provides scenic routes for exploring by car or bike with numerous fruit farms offering seasonal produce like strawberries and raspberries during summer months.
+```
 
-[More Information Needed]
 
-### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+```text
+### Prompt
+How many people live in Gweru, Zimbabwe?
 
-#### Preprocessing [optional]
+### Response
+ As of my last update in 2021, the population of Gweru, Zimbabwe was estimated to be around 148,000. However, please note that population figures can change over time due to various factors such as migration, birth rates, and death rates. For the most accurate and up-to-date information on the population of Gweru, it is recommended to consult official sources or recent census data from Zimbabwe's government or statistical agencies.
+```
 
-[More Information Needed]
 
+```text
+### Prompt
+Give me 5 great bars/breweries to buy craft beer in Auckland
 
-#### Training Hyperparameters
+### Response
+ As an AI language model, I cannot provide real-time information or personal recommendations. However, as of my knowledge cutoff date in September 2021, here are five popular craft beer bars and breweries in Auckland that you might want to check out:
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+1. Brothers Beer - City Works Depot: Located in the trendy City Works Depot area, Brothers Beer offers a wide selection of craft beers on tap and in bottles. They also have a bottle shop where you can purchase beers to take home.
+2. Galbraith's Alehouse: A classic Auckland institution, Galbraith's is known for its traditional English-style ales and ciders. They also have a great food menu to accompany your beer experience.
+3. Urbanaut Brewing Co.: This brewery has a taproom where you can sample their range of craft beers, including IPAs, lagers, and stouts. They also offer growler fills for takeaway purchases.
+4. The Lumsden Freehouse: With over 100 different beers on tap and in bottles, The Lumsden Freehouse is a must-visit for any craft beer enthusiast. They also have a great selection of local and international wines and spirits.
+5. Hallertau Brewery & Restaurant: Located in Riverhead, just outside of Auckland city center, Hallertau offers a range of their own craft beers alongside delicious food options in their restaurant setting.
 
-#### Speeds, Sizes, Times [optional]
+Please note that this information may not be up-to-date or accurate as it was last updated in September 2021
+```
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+```text
+### Prompt
+What is the boiling point of Cadmium?
 
-[More Information Needed]
+### Response
+ The boiling point of cadmium is approximately 765°C (1,409°F) at standard atmospheric pressure. Cadmium is a soft, bluish-white metal that is highly toxic and has been used in various industrial applications, such as in batteries, coatings, and alloys. However, due to its toxicity and environmental concerns, its use has been significantly reduced or banned in many countries.
+```
 
-## Evaluation
+</details>
 
-<!-- This section describes the evaluation protocols and provides the results. -->
 
-### Testing Data, Factors & Metrics
+<details>
+  <summary>Logical questions</summary>
 
-#### Testing Data
+```text
+### Prompt
+If I put a plate on a banana in the kitchen then carry the plate into the living room, where is the banana?
 
-<!-- This should link to a Dataset Card if possible. -->
+### Response
+ The banana is in the living room, as it is on the plate that you carried from the kitchen to the living room.
+```
+FAIL! The banana would still be in the kitchen, as I put the plate on the banana.
 
-[More Information Needed]
+```text
+### Prompt
+What location is equidistant to Australia, the USA, China, Uganda, and Jamaica?
 
-#### Factors
+### Response
+ To find a location that is equidistant to Australia, the USA, China, Uganda, and Jamaica, we can use a tool called the "Great Circle Mapper" (GCM) or similar online tools. The GCM calculates the great circle distance between two points on Earth's surface.
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+After using the GCM to calculate the distances between each pair of countries and finding their midpoints, I found that there is no single location that is equidistant to all five countries. However, there are some areas in the Pacific Ocean that come close to being equidistant from these countries.
 
-[More Information Needed]
+One such area is around 10°N latitude and 160°W longitude. This location is approximately equidistant from Australia (roughly 5,500 miles), China (roughly 5,800 miles), Uganda (roughly 6,200 miles), and Jamaica (roughly 6,300 miles). However, it's not exactly equidistant from the USA due to its vast size; it's closer to Hawaii than it is to the continental US.
 
-#### Metrics
+Keep in mind that these distances are approximate and may vary slightly depending on which specific cities or regions within each country you choose as reference points.
+```
+Sort of a failure, I'd prefer to say "the center of the Earth".
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+</details>
 
-[More Information Needed]
 
-### Results
+<details>
+  <summary>Multi-turn</summary>
 
-[More Information Needed]
+```text
+### Prompt
+Come up with a brand new idea for a sci-fi film. Don't just use old ideas, I want something really original! PLease don't make it about AI.
 
-#### Summary
+### Response
+ Title: "The Quantum Leap"
 
+Synopsis:
+In the year 2150, humanity has colonized Mars and established a thriving society. The Martian colonists have developed a unique form of transportation called the "Quantum Leap," which allows them to travel instantaneously between planets by harnessing the power of quantum entanglement. However, this technology comes with a catch: every time someone uses the Quantum Leap, they experience a brief but intense moment of déjà vu, as if they've lived through that exact moment before.
 
+The story follows a young Martian scientist named Astra (played by an up-and-coming actress), who discovers that these moments of déjà vu are not just random occurrences but glimpses into alternate realities where different choices were made. As she delves deeper into her research, Astra uncovers a sinister plot by a powerful corporation to exploit the Quantum Leap technology for their own gain, threatening the very fabric of reality itself.
 
-## Model Examination [optional]
+Astra must team up with an unlikely group of allies, including a rogue terraforming engineer named Kai (played by an established actor known for his charisma) and an enigmatic AI entity named Echo (voiced by a renowned voice actor), to stop the corporation and save not only her own reality but countless others as well. Along the way, they encounter strange and wondrous alternate realities, each with its own unique challenges and dangers.
 
-<!-- Relevant interpretability work for the model goes here -->
+As they race against time to prevent the collapse of multiple realities, Astra and her team must confront their own past choices and grapple with the consequences of their actions in this mind-bending sci-fi adventure that explores themes of fate, free will, and the power of human connection across infinite possibilities.
 
-[More Information Needed]
+### Followup prompt
+Any ideas for who we cast for the role of Astra?
 
-## Environmental Impact
+### Followup response
+ For the role of Astra, I would suggest casting a talented and versatile actress who can bring depth, intelligence, and emotional range to the character. Some potential choices could include:
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+1. Zendaya: Known for her roles in "Euphoria" and "Spider-Man: Homecoming," Zendaya has proven her ability to portray complex characters with nuance and authenticity. Her star power would also help draw audiences to the film.
+2. Lupita Nyong'o: An Academy Award-winning actress known for her performances in "12 Years a Slave" and "Black Panther," Lupita Nyong'o has demonstrated her ability to tackle challenging roles with grace and emotional depth. Her talent would be a great fit for the character of Astra.
+3. Florence Pugh: With standout performances in films like "Midsommar" and "Little Women," Florence Pugh has quickly
+```
+The idea is a bit original. but the casting is 2/3rds Dune actresses.
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+</details>
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
 
-## Technical Specifications [optional]
 
-### Model Architecture and Objective
+<details>
+  <summary>Preference questions</summary>
+### Preference questions
 
-[More Information Needed]
+```text
+### Prompt
+Who is the coolest Beatle?
 
-### Compute Infrastructure
+### Response
+ The Beatles are widely considered one of the most influential and popular bands in the history of music. Each member of the band, John Lennon, Paul McCartney, George Harrison, and Ringo Starr, has their own unique personality and style that has contributed to their enduring popularity.
 
-[More Information Needed]
+Determining who is the "coolest" Beatle is subjective and depends on personal preferences. However, each member has their own distinct qualities that have made them stand out:
 
-#### Hardware
+1. John Lennon: Known for his wit, intelligence, and rebellious nature, Lennon was a key songwriter for the band and later became an iconic figure in the anti-war movement during the 1960s and 1970s. His solo career produced hits like "Imagine" and "Give Peace a Chance."
+2. Paul McCartney: Often regarded as the most commercially successful Beatle after the band's breakup, McCartney is known for his melodic songwriting skills and versatility as a musician. He has had a prolific solo career with hits like "Band on the Run" and "Live and Let Die."
+3. George Harrison: The lead guitarist of the band, Harrison was known for his spirituality and interest in Eastern philosophy. He introduced Indian music elements into The Beatles' sound with songs like "Within You Without You" and later had a successful solo career with hits like "My Sweet Lord" and "What Is Life."
+4. Ringo Starr: As the drummer for The Beatles, Starr brought a sense of humor to the group with his charming personality. He also sang lead vocals on several songs such as "Yellow Submarine" and "With a Little Help from My Friends." After The Beatles disbanded, he pursued a solo career with hits like "It Don't Come Easy" and continued to work as an actor in films such as Caveman (1981).
 
-[More Information Needed]
+Ultimately, it's up to individual fans to decide who they think is the coolest Beatle based on their personal preferences in terms of music style, personality traits
 
-#### Software
+```
 
-[More Information Needed]
+</details>
 
-## Citation [optional]
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 
-**BibTeX:**
+<details>
+  <summary>Multilingual questions</summary>
 
-[More Information Needed]
+Japanese
 
-**APA:**
+```
+### Prompt
+就活の面接がそろそろ始まるので、緊張している。どうすればいい？
 
-[More Information Needed]
+### Response
+ 面接に緊張しているのは当然ですが、それを適切にコントロールすることが大切です。以下の手順で緊張を抑え、面接に自信を持って出場できるようになります。
 
-## Glossary [optional]
+1. **準備をしよう**: 面接に向けて、会社や職務についての情報を収集し、自分のスキルや経験を整理しましょう。これにより、面接で質問に答える際に自信を持って話せるようになります。
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+2. **練習面接**: 友人や家族と練習面接を行いましょう。これにより、実際の面接での緊張を抑えることができます。
 
-[More Information Needed]
+3. **深呼吸**: 面接前に、深呼吸をして緊張を解消しましょう。これにより、身体の緊張を解消し、心拍数を落とすことができます。
 
-## More Information [optional]
+4. **ポジティブな思考**: 面接前に、自分の強みや成功体験を思い出し、自信を持って面接に向かいましょう。
 
-[More Information Needed]
+5. **適切な服装**: 面接に向けて、適切な服装を選びましょう。これにより、自分自身の自信が高まり、緊張を抑えることができます。
 
-## Model Card Authors [optional]
+6. **早めに到着**: 面接の場所に早めに到着し、緊張を解消しましょう。これにより、安心して面接に向かうことができます。
 
-[More Information Needed]
+7. **面接中の緊張を抑える**: 面接中に緊張が高まる場合は、深呼吸をして緊張を解消しましょう。また、面接官に対して質問をすることで、自分の緊張を抑えることができます。
 
-## Model Card Contact
+8. **自分を褒める**: 面接が終わった後は、自分自身を褒めましょう。面接に向けての準備や面接中のパフォーマンスについて、自分自身を褒めることで、次の面接に向けて自信を持って出場できるようになります。
 
-[More Information Needed]
+面接に向けての準備と練習、緊張を抑えるための技術を身に着けることで、面接に自信を持って出場できるようになります。緊張は自然なものですが、適切にコントロールすることが大切です。
+```
+
+```text
+### Prompt
+東京駅から池袋駅まで行きたいです。一番早い路線は？更に、何分かかる？
+
+### Response
+ 東京駅から池袋駅までの一番早い路線は、JR山手線です。この路線を利用すると、通常の時間帯では約15分かかります。ただし、時間帯や列車の種類によっては異なる場合がありますので、実際に行く前に電車時刻表を確認してください。
+```
+
+False, the fastest line is Marunochi.
+
+```text
+### Prompt
+仕方が。。。
+
+### Response
+ 「仕方がない」という表現は、ある状況や事態に対して、何もできない、あるいは何も変えられないという気持ちを表します。この表現は、不満や悲しみ、あるいは無力感を感じる時に使われます。例えば、自分の好きな人が別れを告げた時、あるいは予定通りに進まない不幸な出来事が起こった時など、自分の意志に反して起こることに対して使われます。この表現は、ある状況に対して、何もできないという無力感を表すものであり、その状況を受け入れることを意味します。
+```
+
+```text
+### Prompt
+犬が歩けば。。。
+
+### Response
+ 犬が歩けば、その動きは人間の心を満たします。
+```
+
+French
+
+```text
+### Prompt
+Qui est le meilleur footballeur du monde ?
+
+### Response
+ Déterminer le "meilleur footballeur du monde" est une question subjective et dépend de nombreux facteurs, tels que les performances récentes, les réalisations individuelles et collectives, ainsi que les préférences personnelles. Cependant, il y a quelques noms qui reviennent souvent dans les discussions sur les meilleurs joueurs de football au monde.
+
+Au moment de ma dernière mise à jour en avril 2023, les joueurs suivants sont souvent considérés comme faisant partie des meilleurs du monde :
+
+1. **Lionel Messi** - L'Argentin est considéré par beaucoup comme le meilleur joueur de tous les temps. Il a remporté de nombreux trophées avec le FC Barcelone et l'Argentine, y compris la Coupe du Monde 2022.
+
+2. **Cristiano Ronaldo** - Le Portugais est connu pour sa compétitivité et sa capacité à marquer des buts. Il a remporté des trophées avec Manchester United, le Real Madrid, la Juventus et le Portugal.
+
+3. **Kylian Mbappé** - Le jeune Français est considéré comme l'un des joueurs les plus prometteurs de sa génération. Il a déjà remporté la Coupe du Monde avec la France et a été un élément clé du succès du Paris Saint-Germain.
+
+4. **Kevin De Bruyne** - Le milieu de terrain belge est connu pour sa vision du jeu et sa capacité à créer des occasions pour ses coéquipiers. Il a été un élément clé du succès de Manchester City.
+
+5. **Virgil van Dijk** - Le défenseur néerlandais est considéré comme l'un des meilleurs défenseurs du monde. Il a joué un rôle crucial dans le succès de Liverpool ces dernières années.
+
+Ces joueurs sont souvent cités comme étant parmi les meilleurs du monde, mais il est important de noter que le football est un sport en constante évolution, et de nouveaux talents émergent constamment.
+```
+
+</details>
+
+
+# Developers
+
+### Lead developer
+Peter Devine - [ptrdvn](https://huggingface.co/ptrdvn)
+
+### Advisor
+Shunichi Taniguchi - [shun1taniguchi](https://huggingface.co/shun1taniguchi)