SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model trained on the dendimaki/v1 dataset that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 26 classes
Training Dataset: dendimaki/v1

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
20	'while the finder feels a deep sense of completeness his or her partner still has a narrativeself that thrives on external validation' 'disassembled' 'location four definitely adds a whole new perspective and can decondition a lot especially if one deepens there but yeah save that for when you feel the timing is good'
26	'i think the emptiness is a different one' 'being like a container for whats arising and the stuff thats arising' 'spaciousness or emptiness'
27	'encased in gelatin' 'feeling full of joy' 'so if i do if i meditate in a certain way i have meditated and it happens and i drop into more of a kind of equalized more still flat perception i would say or just not not perhaps not maybe not flat but its like dropping into a different dimension if you could say that like thats not really its not about the physical that much anymore as much as its a different its like residing in a different field that is more quiet and peaceful and if i sink in in my day to day life i can also go go pretty quickly to that straight away actually but i again i guess i choose not to because again somewhere along the way i think one of my teachers emphasized also feeling the fullness but thats analysis for something else but yeah ive experienced that quite a few times'
18	'mixture of personal and impersonal love' 'it sounds very plausible i think being lonely is one thing if i just sit there in my apartment you know and become more and more still and around boredom or being boring' 'popular term for this change in perception is nonduality or not two'
28	'but the shift into layer four is you know it can be an intense one and it really is very different than everything that comes before it and so you know lots of strange things can happen on the way to it in the direction of it you know sort of associated with it um and its possible that when you felt like you had made progress in that direction and then you had this other sort of experience come in that it was you know just one of those types of things in that direction' 'only reality just unfolding' 'dimensional flatness'
16	'the path of freedom remains emotionless the path of humanity' 'moments and so basically when you come out of the narrative mind you start to fill the mind moments that the narrative mind filled with sensory mind moments and so that can also account for the for the luminosity thing it doesnt necessarily have to be it can be a combination of what you said but when you when you were talking about it i was like oh it could be a mind moment thing just because you know theres more moments of sensory experience in the conscious experience' 'path of humanity'
17	'seer' 'seems like the looker is there looking out your eyes' 'with recalling memories that related to their'
25	'fluid or experiencing one layer' 'layer one level' 'pulled back to probably layer one'
19	'an example of one potential reason relates to personal love for ones child' 'or an all pervasive consciousness' 'it was when my dad died and you know i was like crying but i was like well this is just love so this is okay i wouldnt say this is i want it to stop'
15	'the thing the thing to keep in mind is that for a system for a layer four location four especially but youre sort of close enough you know youre like a hair away from the thing type system what reading those books will do is basically prime you basically primes the system' 'the peace is of a different order than that of any other layer because it is not dependent on any positionality such as i am awareness or i am' 'deeper into layer 4 in later locations the sense of unfolding diminishes until everything feels instantaneous and total '
8	'strong psychological triggers such as the death of a loved one can still cause a reaction in the system but for the most part there is persistent equanimity and joy'
14	'layer 3 can remain accessible in location 4 though usually only the deepest centerless aspects of it' 'dont have that mental abstraction' 'the subjective experience is emmeshed with deep beliefs about what is ultimately real and transitioning to and deepening into location 4 can be disconcerting'
22	'fundamentalist beliefs' 'fundamental wellbeing kind of gets more and more boring in a way' 'curcumin supplement'
3	'the boundaries between work and play blur in location 1 layer 4 each act imbued with purpose and the joy of being' 'in location 1 layer 4 the setting sun doesnt signify an end but a gentle closure a pause for reflection and gratitude' 'i can still get triggered but negative emotions fall off much faster like glimpsing into layer four by doing unprovoked happiness'
4	'memories also tend to arise less because there is an increased focus of attention on the present and because the past is no longer valued as defining the sense of self' 'when youre describing like a deeper nonduality is the absence of layer one'
6	'so you cant stay in location two but youre not able to access the depth of a layout to possibly and certainly layer three that youre able to with your eyes closed' 'cosmic love' 'layer 3 is highly accessible in location 2 however it remains relatively rare for finders to reach layer 3 persistently when they do it is often taken to be end of the path in terms of deepening further into fundamental wellbeing '
21	'psychic intuitive empathic' 'darkness' 'psychedelics'
10	'the main thing was a sense of a kind of strong gravitational pull'
24	'since 2017 was when i did finders course and transitioned'
0	'environment under trigger its more like 11 and then kind of off on my own doing my thing' 'very attached to my mind'
11	'this is partly because one is unable to deepen into it and stabilize in it and partly because it cannot be known objectivelyor even subjectively in the usual sense' 'the unfolding does not happen in anything rather it is total and complete in itself'
1	'only location one layer two seemed to get a graphic and the bird looks a little confused'
9	'feeling like youre dissolving into it' 'in location three there was a certain clarity that i dont have now because it was like less commotion or deadness because like the love would infuse every thought so a thought would come up and instead of me where i am right now i dont want to deal with it it would just be like oh its okay its lets lets just sit with it and the loving feeling would just infuse every thought and then certain judgments that id have oh well i dont really need to look at it that way i can well i can just put love in this or i can just love it so that that id say that was like the most stark contrast'
5	'something into this experience of two so my experience of this has its just now releasing a lot of the as of a couple of days ago thought it might be wise to look at this yeah so ive been experiencing you know this very strange weird nonduality type' 'shifting into layer two' 'things are seen with more distance and objectivity and one typically becomes less reactive the downside of this is that it can be a great place to escape the mind and disassociate from psychological conditioning this is usually whats meant when people speak about spiritual bypassing '
12	'this can lead to a wide range of outcomes from extraordinary life results to some of the amoral behavior observed in late location teachers' 'mind is very quiet' 'essentially this is a metaawareness of what is happening in the mind but there is no sense of being able to engage with it like there is in previous locations '
23	'until youre feeling deeper or more stable in fundamental wellbeing' ' an event in fundamental wellbeing for a while'

Evaluation

Metrics

Label	Accuracy
all	0.4635

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("dendimaki/fewshot-model")
# Run inference
preds = model("pervading presence")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	21.9052	247

Label	Training Sample Count
0	2
1	1
3	5
4	2
5	4
6	11
8	1
9	2
10	1
11	2
12	3
14	4
15	8
16	8
17	11
18	28
19	25
20	14
21	4
22	7
23	2
24	1
25	13
26	30
27	36
28	7

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0017	1	0.252	-
0.0862	50	0.1891	-
0.1724	100	0.1793	-
0.2586	150	0.1848	-
0.3448	200	0.1033	-
0.4310	250	0.0473	-
0.5172	300	0.1213	-
0.6034	350	0.0343	-
0.6897	400	0.0276	-
0.7759	450	0.0262	-
0.8621	500	0.0425	-
0.9483	550	0.0482	-

Framework Versions

Python: 3.10.12
SetFit: 1.0.3
Sentence Transformers: 2.7.0
Transformers: 4.40.1
PyTorch: 2.2.1+cu121
Datasets: 2.19.0
Tokenizers: 0.19.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

dendimaki
/

fewshot-model