metadata

base_model: WhereIsAI/UAE-Large-V1
datasets: []
language:
  - en
library_name: sentence-transformers
license: apache-2.0
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:3474
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      Microsoft Corporation believes that its success is based upon its ability
      to transform to meet the needs of customers. Its growth strategy includes
      innovation across its cloud platforms and services, as well as investing
      in complementary businesses, products, services, and technologies to
      extend and grow its product offerings.
    sentences:
      - >-
        What factors caused the surge in Tesla’s stock prices in the first half
        of 2023?
      - What's Microsoft growth strategy in the cloud computing sector?
      - >-
        How has Microsoft Corporation performed in terms of stock prices over
        the past five years?
  - source_sentence: >-
      Amazon reported the Q3 2023 earnings revealing a 21% year-over-year
      increase in the revenue, which stood at $116.38 billion. Net income
      increased 57% to $6.66 billion, or $13.21 per diluted share, compared to
      $4.23 billion, or $8.42 per diluted share, in third quarter 2022. Amazon
      Web Services (AWS) revenue grew 32% in the quarter to $15 billion.
    sentences:
      - Can you tell about Amazon's Q3 2023 earnings?
      - What was the net income of Microsoft in Fiscal Year 2024?
      - What is the significance of EBITDA in financial analysis?
  - source_sentence: For the fiscal year 2024, Walmart had an operating profit margin of 20%.
    sentences:
      - What is Pfizer's dividend yield for the financial year 2022?
      - >-
        What was Exxon Mobil Corporation's net income for the fourth quarter of
        2023?
      - >-
        What is the operating profit margin for Walmart for the fiscal year
        2024?
  - source_sentence: >-
      The slowdown in construction, particularly in developing markets, resulted
      in a decrease in demand for Caterpillar's machinery and equipment, which
      negatively impacted the revenue for the year 2022.
    sentences:
      - >-
        How did the slow down in construction in 2022 affect Caterpillar's
        revenues?
      - What is JP Morgan's strategy when it comes to sustainability?
      - What was the debt-to-equity ratio for Tesla Inc in Q4 of 2022?
  - source_sentence: >-
      According to Johnson & Johnson’s 2024 guidance report, their
      pharmaceutical sector was projected to grow by 7% in 2023 after
      considering crucial factors like the overall market demand, introduction
      of new drugs and potential impact of patent expirations.
    sentences:
      - >-
        What are Caterpillar's initiatives for enhancing its product
        sustainability?
      - How is JPMorgan Chase & Co. improving its cybersecurity measures?
      - >-
        What was the projected growth of Johnson & Johnson’s pharmaceutical
        sector in 2023?
model-index:
  - name: UAE-Large-V1-financial-embeddings-matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 1024
          type: dim_1024
        metrics:
          - type: cosine_accuracy@1
            value: 0.8316062176165803
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9326424870466321
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.966321243523316
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9896373056994818
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.8316062176165803
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.31088082901554404
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1932642487046632
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09896373056994817
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.8316062176165803
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.9326424870466321
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.966321243523316
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9896373056994818
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9113990251008172
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8860854099843737
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.886565872062324
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.8290155440414507
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9326424870466321
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.966321243523316
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9844559585492227
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.8290155440414507
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.31088082901554404
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1932642487046632
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09844559585492228
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.8290155440414507
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.9326424870466321
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.966321243523316
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9844559585492227
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9098442107332023
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8854439098610082
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8863342112694444
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.8238341968911918
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9378238341968912
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9637305699481865
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9844559585492227
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.8238341968911918
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.3126079447322971
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19274611398963729
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09844559585492228
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.8238341968911918
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.9378238341968912
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9637305699481865
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9844559585492227
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9085199240883707
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8836016530964717
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8844289493397997
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.8212435233160622
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9326424870466321
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.961139896373057
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9792746113989638
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.8212435233160622
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.31088082901554404
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19222797927461138
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09792746113989637
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.8212435233160622
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.9326424870466321
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.961139896373057
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9792746113989638
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9050964679750835
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8807097623159799
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8817273654804927
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.8186528497409327
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9352331606217616
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.961139896373057
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9792746113989638
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.8186528497409327
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.3117443868739206
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19222797927461138
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09792746113989637
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.8186528497409327
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.9352331606217616
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.961139896373057
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9792746113989638
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9031436826413919
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8781797433999506
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8793080516202277
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.7979274611398963
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9222797927461139
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9585492227979274
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9792746113989638
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7979274611398963
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.307426597582038
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19170984455958548
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09792746113989637
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7979274611398963
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.9222797927461139
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9585492227979274
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9792746113989638
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8935743388819871
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8654926391973025
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8667278930244052
            name: Cosine Map@100

UAE-Large-V1-financial-embeddings-matryoshka

This is a sentence-transformers model finetuned from WhereIsAI/UAE-Large-V1. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: WhereIsAI/UAE-Large-V1
Maximum Sequence Length: 512 tokens
Output Dimensionality: 1024 tokens
Similarity Function: Cosine Similarity
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("rbhatia46/UAE-Large-V1-financial-rag-matryoshka")
# Run inference
sentences = [
    'According to Johnson & Johnson’s 2024 guidance report, their pharmaceutical sector was projected to grow by 7% in 2023 after considering crucial factors like the overall market demand, introduction of new drugs and potential impact of patent expirations.',
    'What was the projected growth of Johnson & Johnson’s pharmaceutical sector in 2023?',
    'How is JPMorgan Chase & Co. improving its cybersecurity measures?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_1024
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.8316
cosine_accuracy@3	0.9326
cosine_accuracy@5	0.9663
cosine_accuracy@10	0.9896
cosine_precision@1	0.8316
cosine_precision@3	0.3109
cosine_precision@5	0.1933
cosine_precision@10	0.099
cosine_recall@1	0.8316
cosine_recall@3	0.9326
cosine_recall@5	0.9663
cosine_recall@10	0.9896
cosine_ndcg@10	0.9114
cosine_mrr@10	0.8861
cosine_map@100	0.8866

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.829
cosine_accuracy@3	0.9326
cosine_accuracy@5	0.9663
cosine_accuracy@10	0.9845
cosine_precision@1	0.829
cosine_precision@3	0.3109
cosine_precision@5	0.1933
cosine_precision@10	0.0984
cosine_recall@1	0.829
cosine_recall@3	0.9326
cosine_recall@5	0.9663
cosine_recall@10	0.9845
cosine_ndcg@10	0.9098
cosine_mrr@10	0.8854
cosine_map@100	0.8863

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.8238
cosine_accuracy@3	0.9378
cosine_accuracy@5	0.9637
cosine_accuracy@10	0.9845
cosine_precision@1	0.8238
cosine_precision@3	0.3126
cosine_precision@5	0.1927
cosine_precision@10	0.0984
cosine_recall@1	0.8238
cosine_recall@3	0.9378
cosine_recall@5	0.9637
cosine_recall@10	0.9845
cosine_ndcg@10	0.9085
cosine_mrr@10	0.8836
cosine_map@100	0.8844

Information Retrieval

Dataset: dim_256
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.8212
cosine_accuracy@3	0.9326
cosine_accuracy@5	0.9611
cosine_accuracy@10	0.9793
cosine_precision@1	0.8212
cosine_precision@3	0.3109
cosine_precision@5	0.1922
cosine_precision@10	0.0979
cosine_recall@1	0.8212
cosine_recall@3	0.9326
cosine_recall@5	0.9611
cosine_recall@10	0.9793
cosine_ndcg@10	0.9051
cosine_mrr@10	0.8807
cosine_map@100	0.8817

Information Retrieval

Dataset: dim_128
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.8187
cosine_accuracy@3	0.9352
cosine_accuracy@5	0.9611
cosine_accuracy@10	0.9793
cosine_precision@1	0.8187
cosine_precision@3	0.3117
cosine_precision@5	0.1922
cosine_precision@10	0.0979
cosine_recall@1	0.8187
cosine_recall@3	0.9352
cosine_recall@5	0.9611
cosine_recall@10	0.9793
cosine_ndcg@10	0.9031
cosine_mrr@10	0.8782
cosine_map@100	0.8793

Information Retrieval

Dataset: dim_64
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7979
cosine_accuracy@3	0.9223
cosine_accuracy@5	0.9585
cosine_accuracy@10	0.9793
cosine_precision@1	0.7979
cosine_precision@3	0.3074
cosine_precision@5	0.1917
cosine_precision@10	0.0979
cosine_recall@1	0.7979
cosine_recall@3	0.9223
cosine_recall@5	0.9585
cosine_recall@10	0.9793
cosine_ndcg@10	0.8936
cosine_mrr@10	0.8655
cosine_map@100	0.8667

Training Details

Training Dataset

Unnamed Dataset

Size: 3,474 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 15 tokens
mean: 44.84 tokens
max: 112 tokens

min: 8 tokens
mean: 18.34 tokens
max: 32 tokens

	positive	anchor
type	string	string
details	min: 15 tokens mean: 44.84 tokens max: 112 tokens	min: 8 tokens mean: 18.34 tokens max: 32 tokens

Samples:

positive	anchor
`Exxon Mobil faces substantial risk factors including fluctuating market prices for oil and gas, regulatory environment changes and the potential for catastrophic accidents such as oil spills.`	`What is the key risk factor faced by Exxon Mobil in the energy sector?`
`Tesla’s remarkable revenue growth in 2023 is largely driven by its robust electric vehicle sales in China and the strong demand for its energy storage products.`	`What is the main reason behind Tesla’s revenue growth in 2023?`
`Amazon is expected to see a sales growth of 23% in the next financial year, driven by the increased demand for their ecommerce business and strong growth in AWS. This projection is subject to changes in the market condition and customer spending patterns.`	`What is the projected sales growth for Amazon in the next financial year?`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        1024,
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
gradient_accumulation_steps: 16
learning_rate: 2e-05
num_train_epochs: 4
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 4
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: True
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_1024_cosine_map@100	dim_128_cosine_map@100	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_64_cosine_map@100	dim_768_cosine_map@100
0.8807	6	-	0.8708	0.8499	0.8647	0.8705	0.8307	0.8700
1.4679	10	0.7358	-	-	-	-	-	-
1.9083	13	-	0.8848	0.8724	0.8782	0.8861	0.8617	0.8855
2.9358	20	0.1483	0.8865	0.8793	0.8814	0.8857	0.8667	0.8863
3.5229	24	-	0.8866	0.8793	0.8817	0.8844	0.8667	0.8863

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.6
Sentence Transformers: 3.0.1
Transformers: 4.41.2
PyTorch: 2.1.2+cu121
Accelerate: 0.32.1
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}