Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+ "word_embedding_dimension": 768,
+ "pooling_mode_cls_token": false,
+ "pooling_mode_mean_tokens": true,
+ "pooling_mode_max_tokens": false,
+ "pooling_mode_mean_sqrt_len_tokens": false,
+ "pooling_mode_weightedmean_tokens": false,
+ "pooling_mode_lasttoken": false,
+ "include_prompt": true
+}

README.md

@@ -0,0 +1,264 @@
+---
+base_model: sentence-transformers/all-mpnet-base-v2
+library_name: setfit
+metrics:
+- accuracy
+pipeline_tag: text-classification
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+widget:
+- text: 'John Ondespot Help me out. So Yellen has to tell the President that they
+ cannot afford to pay bondholders in the favour of US civil servants and military
+ and homeless to keep society rolling and let the big banks hold out for money
+ down the line? To float the entire USA financial system from collapse but also
+ from societal rioting on Capitol Hill? I am getting this? Cause the more I read
+ this is quite a debt watched by the major credit leaders of the US commercial
+ and credit banking system?
+
+ '
+- text: 'Independent I disagree that, in your words, Lula "is the biggest thief in
+ Brazil''s history." The excellent Guardian article you cite requires a careful
+ reading to the end. To me, it seems like the Brazilian parliamentary system practically
+ encourages corruption and has been rife with corruption in most administrations. Lula
+ too fell into corruption to gain political support to enact his social reforms
+ when faced with a minority in Congress. (This reminds me of the leftist Peruvian
+ president who tried to dissolve the conservative dominated Congress that block
+ any of his reforms.) Lula resorted to bribes to get support from minority parties.
+ From the Guardian article: "Although illegal, this allowed the Workers’ Party
+ to get things done. Lula’s first term delivered impressive progress on alleviating
+ poverty, social spending and environmental controls."At the same time, "it was
+ the Workers’ Party that had put in place the judicial reforms that allowed the
+ investigation to go ahead. There would have been no Car Wash if the government
+ had not appointed, in September 2013, an independent attorney general."So maybe
+ Lula will prove to be a better president today.
+
+ '
+- text: 'The reality is that in Brazil the level of corruption has exceeded all limits,
+ our system is similar to the American one, but imagine that a former president
+ convicted of corruption in which he should have served a sentence of 9 years in
+ 2018 was released for cheating by the judiciary and could still run for office
+ (which is illegal under our constitution).Lula is not just a communist, he is
+ the "kingpin" these protests are a sample of the desperation of people who fear
+ for their freedom and integrity.
+
+ '
+- text: 'The ‘Trump of the Tropics’ Goes Bust The definitive challenge for Luiz Inácio
+ Lula da Silva: to be president for all the people. SÃO PAULO, Brazil — As a shocked
+ nation watched live on television and social media, thousands of radical supporters
+ of a defeated president marched on the seat of the federal government, convinced
+ that an election had been stolen. The mob ransacked the Congress, the Supreme
+ Court and the presidential palace. It took the authorities several hours to arrest
+ hundreds of people and finally restore order. The definitive challenge for Luiz
+ Inácio Lula da Silva: to be president for all the people.
+
+ '
+- text: 'Friends,Speaker McCarthy and Representative Taylor Greene aren''t the problems---WE
+ ARE!!!! And, by we, I mean the people who registered and voted for them. These
+ clowns aren''t in the House of Representatives by osmosis, our fellow citizens
+ voted them into office. Obviously, some Americans want the US to be run this way.
+ But if you don''t, you can do something about it. Find out who''s going to be
+ running for office in your area (county, city, state, federal) and start asking
+ them questions? Are they running to represent you or someone else? Go ahead and
+ ask them personal questions, tell them you read about it on "deepfake" website.
+ But more importantly, don''t complain online. You can do something to stop them.
+ It''s a simple 4 step process: 1) Clean out your ears! 2) Support the people you
+ think will actually help you. 3) Register and 4) Vote. Yes, vote. Vote it like
+ my life depends on it because it does!
+
+ '
+inference: true
+model-index:
+- name: SetFit with sentence-transformers/all-mpnet-base-v2
+ results:
+ - task:
+ type: text-classification
+ name: Text Classification
+ dataset:
+ name: Unknown
+ type: unknown
+ split: test
+ metrics:
+ - type: accuracy
+ value: 1.0
+ name: Accuracy
+---
+
+# SetFit with sentence-transformers/all-mpnet-base-v2
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 384 tokens
+- **Number of Classes:** 2 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label | Examples |
+|:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| yes | <ul><li>"NYT.1/1/2023. As Lula Becomes Brazil's President, Bolsonaro Flees to Florida.Kudos to the NYT journalism for a first-rate article about the chaotic and surrealistic end of the ex-military president Bolsonaro's administration. Among his many policy mistakes, some described as of criminal nature, the death of his political career was to escape the country before passing the presidential sash to President Lula. Bolsonaro is lucky to be a politician and no longer a military man. For an army officer to flee from a combat theater leaving behind his comrades, is a court martial offense. One thing is for sure. He destroyed any hope of the Brazilian military to one day return to power. Moreover, President Lula's success or failure depends on how his administration deals with the economy rather than on political opposition from Bolsonaro that from Orlando or Rio de Janeiro will fade away.\n"</li><li>'A few days ago I listened to an interview with the left-of-center new President of Brazil, Luiz Inácio Lula da Silva. He said education, health care and food for poor people aren’t cost, but investments.How I wish American legislatures would think like him.\n'</li><li>'After the dictatorship there was a blanket pardon. No military men was ever prosecuted for the assassinations, torture, rapes committed in the name of the government. Lula said he will be the president for all Brazilians, including the ones who did not vote for him. He said it was time to reach out in the families and end divisions. But he said he will prosecute crimes of the previous administration. He is correct. Brazil lost (proportionally) more people than any other country to COVID. A country thst has been a leader and an example in mass vaccinations. The hundreds of thousands who died did not need to die. And they should not be hidden under the carpet as if nothing happened.\n'</li></ul> |
+| no | <ul><li>'rivvir No, they didn\'t just want to "die in a war," they also didn\'t want to kill other people they have no reason to kill in some utterly immoral war...that\'s a far cry from the "same danger" as being "poor and desperate."Also, while the journey north has it perils for sure, have a look at the Rio Grande in a southern climate, then look at the Bearing Sea in fall weather!\n'</li><li>'"Spectacle produced fame, which produced power, which produced influence and possibly control." Yes, indeed. And since the Republicans have nothing to sell BUT spectacle -- because "more tax breaks for the wealthy" somehow doesn\'t get sufficient votes from the hoi polloi -- they kept offering it and the hoi polloi (or about a third of us) kept buying it, and now they\'re caught in their own trap. They created the monster that\'s taken control from them.\n'</li><li>"While undoubtedly all this is true, the recent layoffs are different than most. Because what we have is companies, some of the richest in the world, laying off many thousands of employees even though they continue to be profitable. So the ask of managers is difficult. It's not just look the person in the eye. It is: look the person in the eye and tell them that the company to which they'll loyally devoted many years of service has decided to make them unemployed, not out of necessity, not because the company is at risk, but so that some greedy shareholders can earn a few more pennies. They would be asking the manager to defend the indefensible. And if the manager doesn't agree with the lay-offs, it puts them in a very awkward position. Should they resign in disgust (and so one more person without a way to feed their family or pay their mortgage)? Or should they at least tell the employee they don't agree (but what consequences could this have for them if word gets back to their superiors)? Or should they pretend to agree that this appalling, cynical lay-off is somehow appropriate and just a measured, proportionate response to the fact that some activist shareholder only earned $3.2 billion this year? Somehow, while it is totally wrong, it also feels appropriate that these most cynical and inhumane of lay-offs be executed in the most cynical inhumane way.\n"</li></ul> |
+
+## Evaluation
+
+### Metrics
+| Label | Accuracy |
+|:--------|:---------|
+| **all** | 1.0 |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("davidadamczyk/setfit-model-7")
+# Run inference
+preds = model("John Ondespot Help me out. So Yellen has to tell the President that they cannot afford to pay bondholders in the favour of US civil servants and military and homeless to keep society rolling and let the big banks hold out for money down the line? To float the entire USA financial system from collapse but also from societal rioting on Capitol Hill? I am getting this? Cause the more I read this is quite a debt watched by the major credit leaders of the US commercial and credit banking system?
+")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median | Max |
+|:-------------|:----|:-------|:----|
+| Word count | 23 | 107.2 | 272 |
+
+| Label | Training Sample Count |
+|:------|:----------------------|
+| no | 18 |
+| yes | 22 |
+
+### Training Hyperparameters
+- batch_size: (16, 16)
+- num_epochs: (1, 1)
+- max_steps: -1
+- sampling_strategy: oversampling
+- num_iterations: 120
+- body_learning_rate: (2e-05, 2e-05)
+- head_learning_rate: 2e-05
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- l2_weight: 0.01
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0017 | 1 | 0.3073 | - |
+| 0.0833 | 50 | 0.1154 | - |
+| 0.1667 | 100 | 0.0012 | - |
+| 0.25 | 150 | 0.0002 | - |
+| 0.3333 | 200 | 0.0002 | - |
+| 0.4167 | 250 | 0.0001 | - |
+| 0.5 | 300 | 0.0001 | - |
+| 0.5833 | 350 | 0.0001 | - |
+| 0.6667 | 400 | 0.0001 | - |
+| 0.75 | 450 | 0.0001 | - |
+| 0.8333 | 500 | 0.0001 | - |
+| 0.9167 | 550 | 0.0001 | - |
+| 1.0 | 600 | 0.0001 | - |
+
+### Framework Versions
+- Python: 3.10.13
+- SetFit: 1.1.0
+- Sentence Transformers: 3.0.1
+- Transformers: 4.45.2
+- PyTorch: 2.4.0+cu124
+- Datasets: 2.21.0
+- Tokenizers: 0.20.0
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+ doi = {10.48550/ARXIV.2209.11055},
+ url = {https://arxiv.org/abs/2209.11055},
+ author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+ keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+ title = {Efficient Few-Shot Learning Without Prompts},
+ publisher = {arXiv},
+ year = {2022},
+ copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,24 @@
+{
+ "_name_or_path": "sentence-transformers/all-mpnet-base-v2",
+ "architectures": [
+ "MPNetModel"
+ ],
+ "attention_probs_dropout_prob": 0.1,
+ "bos_token_id": 0,
+ "eos_token_id": 2,
+ "hidden_act": "gelu",
+ "hidden_dropout_prob": 0.1,
+ "hidden_size": 768,
+ "initializer_range": 0.02,
+ "intermediate_size": 3072,
+ "layer_norm_eps": 1e-05,
+ "max_position_embeddings": 514,
+ "model_type": "mpnet",
+ "num_attention_heads": 12,
+ "num_hidden_layers": 12,
+ "pad_token_id": 1,
+ "relative_attention_num_buckets": 32,
+ "torch_dtype": "float32",
+ "transformers_version": "4.45.2",
+ "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+ "__version__": {
+ "sentence_transformers": "3.0.1",
+ "transformers": "4.45.2",
+ "pytorch": "2.4.0+cu124"
+ },
+ "prompts": {},
+ "default_prompt_name": null,
+ "similarity_fn_name": null
+}

config_setfit.json

@@ -0,0 +1,7 @@
+{
+ "normalize_embeddings": false,
+ "labels": [
+ "no",
+ "yes"
+ ]
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e0f9688faea23948459ab90568a8d0cb74e9b4d09e476d04ff2a8edcfc8a9ad
+size 437967672

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28f7610a9921eb077b641a5b463c4e841f7890bff0a61eafdea2888cc978851c
+size 7023

modules.json

@@ -0,0 +1,20 @@
+[
+ {
+ "idx": 0,
+ "name": "0",
+ "path": "",
+ "type": "sentence_transformers.models.Transformer"
+ },
+ {
+ "idx": 1,
+ "name": "1",
+ "path": "1_Pooling",
+ "type": "sentence_transformers.models.Pooling"
+ },
+ {
+ "idx": 2,
+ "name": "2",
+ "path": "2_Normalize",
+ "type": "sentence_transformers.models.Normalize"
+ }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+ "max_seq_length": 384,
+ "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+ "bos_token": {
+ "content": "<s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "cls_token": {
+ "content": "<s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "eos_token": {
+ "content": "</s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "mask_token": {
+ "content": "<mask>",
+ "lstrip": true,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "pad_token": {
+ "content": "<pad>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "sep_token": {
+ "content": "</s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "unk_token": {
+ "content": "[UNK]",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ }
+}

tokenizer_config.json

@@ -0,0 +1,72 @@
+{
+ "added_tokens_decoder": {
+ "0": {
+ "content": "<s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "1": {
+ "content": "<pad>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "2": {
+ "content": "</s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "3": {
+ "content": "<unk>",
+ "lstrip": false,
+ "normalized": true,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "104": {
+ "content": "[UNK]",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "30526": {
+ "content": "<mask>",
+ "lstrip": true,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ }
+ },
+ "bos_token": "<s>",
+ "clean_up_tokenization_spaces": false,
+ "cls_token": "<s>",
+ "do_lower_case": true,
+ "eos_token": "</s>",
+ "mask_token": "<mask>",
+ "max_length": 128,
+ "model_max_length": 384,
+ "pad_to_multiple_of": null,
+ "pad_token": "<pad>",
+ "pad_token_type_id": 0,
+ "padding_side": "right",
+ "sep_token": "</s>",
+ "stride": 0,
+ "strip_accents": null,
+ "tokenize_chinese_chars": true,
+ "tokenizer_class": "MPNetTokenizer",
+ "truncation_side": "right",
+ "truncation_strategy": "longest_first",
+ "unk_token": "[UNK]"
+}