First-author system paper on multi-label emotion detection. A wide sweep across state-of-the-art sentence encoders and classical classifiers, against large prompt-based models. The specialised encoders did not win, which was not the result we expected.
Role First author. Venue SemEval-2025, affiliated with ACL. Scope Model architecture decisions, the full inference sweep, and the write-up.
SemEval Task 11 asks a system to detect multiple emotions present in a piece of text at once. Our entry tested a specific and practically important question: whether a well-chosen sentence encoder with a classical classifier on top can match a large prompt-based model on this task.
The appeal of the encoder route is obvious. It is faster, an order of magnitude smaller, cheaper to serve, and it is the architecture the field designed specifically for classification.
It did not win. Large prompt-based models with far more parameters still came out ahead, and the paper reports that rather than hunting for a framing that made our preferred approach look better.
Multi-label emotion detection is harder than it sounds, for a reason worth stating precisely: emotions in text co-occur and correlate, so the labels are not independent. A sentence can be angry and afraid at once, and those two labels are far more likely to appear together than angry and joyful.
That structure is exactly what a classifier sitting on top of a fixed embedding struggles to exploit, because the embedding was produced without knowledge of the label space. It is also exactly what a large model with a good prior over language handles naturally, since it has read enough text to know that fear and anger travel together.
Stating it that way, the result is less surprising after the fact than it was before. That is worth admitting, because a lot of the value of running the sweep was learning to predict which way it would go.
The design was a grid: every encoder against every classifier, evaluated identically.
| Encoders tested | Why it was in the running |
|---|---|
| BGE | Strong general-purpose retrieval embeddings, widely used baseline |
| Jina v3 | Recent, task-conditioned embeddings, designed for adaptability |
| multilingual E5 | Strong multilingual coverage, relevant given the task’s languages |
| XLM-RoBERTa | The established multilingual encoder, and the conservative choice |
| Classifiers tested | Why it was in the running |
|---|---|
| Support vector classifier | Strong on small, high-dimensional feature spaces |
| Logistic regression | The honest baseline; if it wins, the embedding did the work |
| CatBoost | Gradient boosting, robust without much tuning |
| XGBoost | The other gradient boosting standard, for comparability |
Running every combination rather than a promising subset is the part I would defend. A partial sweep tends to confirm whatever the team already believed, because the combinations that get tried are the ones somebody expected to work.
Each encoder is a fixed feature extractor rather than something fine tuned, so a piece of text becomes one vector and that vector is all the classifier ever sees. BGE, Jina v3, multilingual E5 and XLM-RoBERTa each produce their own representation of the same texts. The four classifiers, support vector classifier, logistic regression, CatBoost and XGBoost, are then fitted on whichever representation they are paired with. That is what makes the sixteen cells comparable: between any two cells exactly one thing changes, the encoder or the classifier, never the data and never the protocol.
The output side is where multi label matters. The task allows several emotions on one piece of text, so the head is not a single choice among labels but a separate decision per label, and every cell of the grid has to emit that same shape of prediction to be scored the same way. It is also the reason the fixed embedding is at a disadvantage: the vector was computed with no view of the label space, so whatever correlation exists between anger and fear has to be recovered downstream by a classifier looking at one label at a time.
The comparison arm sits outside the grid entirely. Large prompt based models are given the same texts and asked for the same label set, then pushed through the identical evaluation, so the two families meet at one measurement rather than at two separately tuned pipelines. The rest is orchestration: mixed precision to fit the larger encoders into available memory, shell scripts driving combinations unattended across cloud instances, and VRAM monitoring to catch runs that would otherwise have died silently overnight.
Read the figure below as two arms meeting. The thing to look for is the single evaluation block that both the grid and the prompt based arm have to pass through.
Prompt-based models with substantially more parameters outperformed the specialised encoder pipelines.
This is uncomfortable for a reason beyond the leaderboard. The encoder plus classifier design is the efficient one, the deployable one, and the one an engineer with a latency budget wants to be true. Reporting that it lost is reporting against the interest of the approach I would rather use in production.
The honest reading is narrower than “scale wins”. What the result shows is that on this task, at these sizes, with the label structure this task has, the general prior beat the specialised architecture. A task with independent labels, or far more training data, could easily go the other way, and I would not extrapolate from one task to a rule.
The experimental design is the interesting part of the paper. The engineering is the reason it exists at all.
Sixteen combinations, across multiple large encoders, over the full dataset, is a lot of inference. It ran on mixed precision to fit larger encoders into available memory, with shell-scripted orchestration across cloud instances so that combinations ran unattended, and with VRAM monitoring to catch the runs that would otherwise have died silently overnight and been discovered the next morning.
None of that is publishable. All of it is the difference between a complete grid and a convenient subset, and the completeness is what makes the negative result trustworthy.
Published at SemEval-2025, affiliated with ACL. First author.