Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,9 @@
+{
+ "word_embedding_dimension": 768,
+ "pooling_mode_cls_token": false,
+ "pooling_mode_mean_tokens": true,
+ "pooling_mode_max_tokens": false,
+ "pooling_mode_mean_sqrt_len_tokens": false,
+ "pooling_mode_weightedmean_tokens": false,
+ "pooling_mode_lasttoken": false
+}

README.md

@@ -0,0 +1,225 @@
+---
+library_name: setfit
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+metrics:
+- accuracy
+widget:
+- text: Specific information applicable to Parties, including regional economic integration
+ organizations and their member States, that have reached an agreement to act jointly
+ under Article 4, paragraph 2, of the Paris Agreement, including the Parties that
+ agreed to act jointly and the terms of the agreement, in accordance with Article
+ 4, paragraphs 16–18, of the Paris Agreement. Not applicable. (c). How the Party’s
+ preparation of its nationally determined contribution has been informed by the
+ outcomes of the global stocktake, in accordance with Article 4, paragraph 9, of
+ the Paris Agreement.
+- text: 'In the shipping and aviation sectors, emission reduction efforts will be
+ focused on distributing eco-friendly ships and enhancing the operational efficiency
+ of aircraft. Agriculture, livestock farming and fisheries: The Republic Korea
+ is introducing various options to accelerate low-carbon farming, for instance,
+ improving irrigation techniques in rice paddies and adopting low-input systems
+ for nitrogen fertilizers.'
+- text: As part of this commitment, Oman s upstream oil and gas industry is developing
+ economically viable solutions to phase out routine flaring as quickly as possible
+ and ahead of the World Bank s target date. IV. Climate Preparedness and Resilience.
+ The Sultanate of Oman has stepped up its efforts in advancing its expertise and
+ methodologies to better manage the climate change risks over the past five years.
+ The adaptation efforts are underway, and the status of adaptation planning is
+ still at a nascent stage.
+- text: 'Synergy and coherence 46 VII- Gender and youth 46 VIII- Education and employment
+ 48 ANNEXES. 49 Annex No. 1: Details of mitigation measures, conditional and non-conditional,
+ by sector 49 Annex No.2: List of adaptation actions proposed by sectors. 57 Annex
+ No.3: GCF project portfolio. 63 CONTRIBUTION DENTERMINEE AT NATIONAL LEVEL CDN
+ MAURITANIE LIST OF TABLES Table 1: Summary of funding needs for the CND 2021-2030
+ updated. 12 Table 2: CND 2021-2030 mitigation measures updated by sector (cumulative
+ cost and reduction potential for the period). 14 Table 3: CND 2021-2030 adaptation
+ measures updated by sector. Error!'
+- text: In the transport sector, restructuing is planned through a number of large
+ infrastructure initiatives aiming to revive the role of public transport and achieving
+ a relevant share of fuel efficient vehicles. Under both the conditional and unconditional
+ mitigation scenarios, Lebanon will achieve sizeable emission reductions. With
+ regards to adaptation, Lebanon has planned comprehensive sectoral actions related
+ to water, agriculture/forestry and biodiversity, for example related to irrigation,
+ forest management, etc. It also continues developing adaptation strategies in
+ the remaining sectors.
+pipeline_tag: text-classification
+inference: false
+co2_eq_emissions:
+ emissions: 25.8151164022705
+ source: codecarbon
+ training_type: fine-tuning
+ on_cloud: false
+ cpu_model: Intel(R) Xeon(R) CPU @ 2.00GHz
+ ram_total_size: 12.674781799316406
+ hours_used: 0.622
+ hardware_used: 1 x Tesla T4
+base_model: ppsingh/SECTOR-multilabel-mpnet_w
+---
+
+# SetFit with ppsingh/SECTOR-multilabel-mpnet_w
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [ppsingh/SECTOR-multilabel-mpnet_w](https://huggingface.co/ppsingh/SECTOR-multilabel-mpnet_w) as the Sentence Transformer embedding model. A [SetFitHead](huggingface.co/docs/setfit/reference/main#setfit.SetFitHead) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [ppsingh/SECTOR-multilabel-mpnet_w](https://huggingface.co/ppsingh/SECTOR-multilabel-mpnet_w)
+- **Classification head:** a [SetFitHead](huggingface.co/docs/setfit/reference/main#setfit.SetFitHead) instance
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Classes:** 4 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("ppsingh/iki_sector_setfit")
+# Run inference
+preds = model("In the shipping and aviation sectors, emission reduction efforts will be focused on distributing eco-friendly ships and enhancing the operational efficiency of aircraft. Agriculture, livestock farming and fisheries: The Republic Korea is introducing various options to accelerate low-carbon farming, for instance, improving irrigation techniques in rice paddies and adopting low-input systems for nitrogen fertilizers.")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median | Max |
+|:-------------|:----|:-------|:----|
+| Word count | 35 | 76.164 | 170 |
+
+### Training Hyperparameters
+- batch_size: (16, 2)
+- num_epochs: (1, 0)
+- max_steps: -1
+- sampling_strategy: oversampling
+- body_learning_rate: (2e-05, 1e-05)
+- head_learning_rate: 0.01
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.01
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0005 | 1 | 0.2029 | - |
+| 0.0993 | 200 | 0.0111 | 0.1124 |
+| 0.1985 | 400 | 0.0063 | 0.111 |
+| 0.2978 | 600 | 0.0183 | 0.1214 |
+| 0.3970 | 800 | 0.0197 | 0.1248 |
+| 0.4963 | 1000 | 0.0387 | 0.1339 |
+| 0.5955 | 1200 | 0.0026 | 0.1181 |
+| 0.6948 | 1400 | 0.0378 | 0.1208 |
+| 0.7940 | 1600 | 0.0285 | 0.1267 |
+| 0.8933 | 1800 | 0.0129 | 0.1254 |
+| 0.9926 | 2000 | 0.0341 | 0.1271 |
+
+### Environmental Impact
+Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
+- **Carbon Emitted**: 0.026 kg of CO2
+- **Hours Used**: 0.622 hours
+
+### Training Hardware
+- **On Cloud**: No
+- **GPU Model**: 1 x Tesla T4
+- **CPU Model**: Intel(R) Xeon(R) CPU @ 2.00GHz
+- **RAM Size**: 12.67 GB
+
+### Framework Versions
+- Python: 3.10.12
+- SetFit: 1.0.3
+- Sentence Transformers: 2.3.1
+- Transformers: 4.35.2
+- PyTorch: 2.1.0+cu121
+- Datasets: 2.3.0
+- Tokenizers: 0.15.1
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+ doi = {10.48550/ARXIV.2209.11055},
+ url = {https://arxiv.org/abs/2209.11055},
+ author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+ keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+ title = {Efficient Few-Shot Learning Without Prompts},
+ publisher = {arXiv},
+ year = {2022},
+ copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,65 @@
+{
+ "_name_or_path": "ppsingh/SECTOR-multilabel-mpnet_w",
+ "architectures": [
+ "MPNetModel"
+ ],
+ "attention_probs_dropout_prob": 0.1,
+ "bos_token_id": 0,
+ "eos_token_id": 2,
+ "hidden_act": "gelu",
+ "hidden_dropout_prob": 0.1,
+ "hidden_size": 768,
+ "id2label": {
+ "0": "Agriculture",
+ "1": "Buildings",
+ "2": "Coastal Zone",
+ "3": "Cross-Cutting Area",
+ "4": "Disaster Risk Management (DRM)",
+ "5": "Economy-wide",
+ "6": "Education",
+ "7": "Energy",
+ "8": "Environment",
+ "9": "Health",
+ "10": "Industries",
+ "11": "LULUCF/Forestry",
+ "12": "Social Development",
+ "13": "Tourism",
+ "14": "Transport",
+ "15": "Urban",
+ "16": "Waste",
+ "17": "Water"
+ },
+ "initializer_range": 0.02,
+ "intermediate_size": 3072,
+ "label2id": {
+ "Agriculture": 0,
+ "Buildings": 1,
+ "Coastal Zone": 2,
+ "Cross-Cutting Area": 3,
+ "Disaster Risk Management (DRM)": 4,
+ "Economy-wide": 5,
+ "Education": 6,
+ "Energy": 7,
+ "Environment": 8,
+ "Health": 9,
+ "Industries": 10,
+ "LULUCF/Forestry": 11,
+ "Social Development": 12,
+ "Tourism": 13,
+ "Transport": 14,
+ "Urban": 15,
+ "Waste": 16,
+ "Water": 17
+ },
+ "layer_norm_eps": 1e-05,
+ "max_position_embeddings": 514,
+ "model_type": "mpnet",
+ "num_attention_heads": 12,
+ "num_hidden_layers": 12,
+ "pad_token_id": 1,
+ "problem_type": "multi_label_classification",
+ "relative_attention_num_buckets": 32,
+ "torch_dtype": "float32",
+ "transformers_version": "4.35.2",
+ "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,7 @@
+{
+ "__version__": {
+ "sentence_transformers": "2.3.1",
+ "transformers": "4.35.2",
+ "pytorch": "2.1.0+cu121"
+ }
+}

config_setfit.json

@@ -0,0 +1,9 @@
+{
+ "normalize_embeddings": true,
+ "labels": [
+ "Economy-wide",
+ "Energy",
+ "Other Sector",
+ "Transport"
+ ]
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5f137bb2e1e7da1eebf8b21d4b5878675c41b4240614b3e4bccb248029eb52c
+size 437967672

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88559af6420967265b7519c9b35c1a3efa8a7ef3ee3c4b40d3f5f3225ffab36b
+size 13858

modules.json

@@ -0,0 +1,14 @@
+[
+ {
+ "idx": 0,
+ "name": "0",
+ "path": "",
+ "type": "sentence_transformers.models.Transformer"
+ },
+ {
+ "idx": 1,
+ "name": "1",
+ "path": "1_Pooling",
+ "type": "sentence_transformers.models.Pooling"
+ }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+ "max_seq_length": 512,
+ "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+ "bos_token": {
+ "content": "<s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "cls_token": {
+ "content": "<s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "eos_token": {
+ "content": "</s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "mask_token": {
+ "content": "<mask>",
+ "lstrip": true,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "pad_token": {
+ "content": "<pad>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "sep_token": {
+ "content": "</s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "unk_token": {
+ "content": "[UNK]",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ }
+}

tokenizer_config.json

@@ -0,0 +1,73 @@
+{
+ "added_tokens_decoder": {
+ "0": {
+ "content": "<s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "1": {
+ "content": "<pad>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "2": {
+ "content": "</s>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "3": {
+ "content": "<unk>",
+ "lstrip": false,
+ "normalized": true,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "104": {
+ "content": "[UNK]",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "30526": {
+ "content": "<mask>",
+ "lstrip": true,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ }
+ },
+ "bos_token": "<s>",
+ "clean_up_tokenization_spaces": true,
+ "cls_token": "<s>",
+ "do_lower_case": true,
+ "eos_token": "</s>",
+ "mask_token": "<mask>",
+ "max_length": 128,
+ "model_max_length": 512,
+ "pad_to_multiple_of": null,
+ "pad_token": "<pad>",
+ "pad_token_type_id": 0,
+ "padding_side": "right",
+ "problem_type": "multi_label_classification",
+ "sep_token": "</s>",
+ "stride": 0,
+ "strip_accents": null,
+ "tokenize_chinese_chars": true,
+ "tokenizer_class": "MPNetTokenizer",
+ "truncation_side": "right",
+ "truncation_strategy": "longest_first",
+ "unk_token": "[UNK]"
+}