Twitter humor research combines human judgments, naturally occurring reactions, and machine-learning benchmarks. The available studies range from 27,000 Spanish tweets to 89 million historical tweets used to provide author context. Together, these figures show how researchers measure humor, sarcasm, offense, and the difficulty of interpreting short social posts.
Table of contents
- How Twitter humor is measured
- Large humor and reaction-labeled collections
- Spanish humor corpus
- Sarcasm, contributors, and tweet length
- Personalized sarcasm datasets
- Model performance and benchmark results
How Twitter humor is measured
One of the clearest patterns in Twitter humor statistics is that “humor” is not a single measurement. Some projects ask people to label a post. Others use a user’s reaction as an available signal. Sarcasm research may distinguish what an author intended from what a reader perceived. These choices produce datasets that answer different questions.
SemEval 2021 Task 7, titled HaHackathon: Detecting and Rating Humor and Offense, collected 10,000 texts for humor and offense annotation. The collection included Twitter data, but it also mixed Twitter with the Kaggle Short Jokes dataset. Each item was annotated by 20 annotators, whose ages ranged from 18 to 70. The shared task launched four subtasks and attracted between 36 and 58 submissions across them.
That setup matters for interpreting results. A score from a mixed Twitter-and-jokes benchmark is not automatically a platform-wide estimate of how funny Twitter users are. It is a measurement of performance on a defined research collection, with a specific annotation process and age range.
The Spanish humor research used a different scale and language. A Crowd-Annotated Spanish Corpus for Humor Analysis contained 27,000 Spanish tweets. Each tweet received about four annotations, and 1,300 people contributed tags. The corpus split tweets evenly between humorous and non-humorous accounts. The equal split is useful for comparison, but it should not be read as the natural proportion of humorous posts on Twitter.
Another benchmark, #HashtagWars: Learning a Sense of Humor, reported a best supervised accuracy of 63.7%. Accuracy is a classification measure, so this result belongs to that task’s labels and evaluation design rather than to a general estimate of humor recognition across all Twitter content.
Large humor and reaction-labeled collections
Manual annotation is detailed, but it can be expensive. CHoRaL, described in CHoRaL: Collecting Humor Reaction Labels from Millions of Social Media Users, labeled 785K posts related to COVID-19 for humor. The project used naturally available user reactions rather than manual annotation for labeling.
The contrast with SemEval is substantial. SemEval assigned 20 annotators to each item in a 10,000-text collection. CHoRaL worked with 785K COVID-19-related posts and used reactions that were already available from users. Those are different forms of evidence: one records structured judgments from a recruited annotation process, while the other treats user behavior as a labeling signal.
| Research collection | Quantified scale | Labeling or task detail |
|---|---|---|
| SemEval 2021 Task 7 | 10,000 texts | 20 annotators per item |
| CHoRaL | 785K posts | Natural user reactions |
| Spanish humor corpus | 27,000 tweets | About 4 annotations per tweet |
| iSarcasm | 4,484 tweets | Author-labeled sarcasm data |
| Personalized Twitter sarcasm dataset | 35K labeled tweets | 33K different authors |
The CHoRaL figure is specifically tied to posts related to COVID-19. It therefore describes a topical collection, not all humor posts or all Twitter posts. The use of reactions also means that the label reflects how users responded under that project’s collection conditions. The study’s contribution is scale and naturally available feedback, while the annotation-based corpora provide more explicitly requested judgments.
Spanish humor corpus
The Spanish corpus provides several useful statistics about how a humor dataset can be assembled. Its 27,000 tweets were gathered in Spanish, and 1,300 people supplied the tags. With about four annotations per tweet, the corpus combines a large number of short texts with multiple judgments for each example.
Its account-level design also deserves attention: tweets were split evenly between humorous and non-humorous accounts. That gives the corpus a balanced structure for a humor classification task. However, “evenly split” describes the research sample. It does not establish that half of Twitter accounts, half of Spanish tweets, or half of social-media activity is humorous.
The #HashtagWars result adds a performance reference point. Its best supervised accuracy was 63.7%. Because the statistic is an accuracy value from a named supervised task, it can be compared with other results only when the labels, split, language, and evaluation procedure are also comparable. A single accuracy percentage does not reveal whether a system was equally successful on humorous and non-humorous examples.
These details illustrate why source labels belong next to the numbers. “27,000 Spanish tweets,” “about four annotations,” and “63.7% accuracy” are all precise facts, but they describe different parts of the research process: corpus size, annotation density, and model evaluation.
Sarcasm, contributors, and tweet length
The iSarcasm dataset, described in iSarcasm: A Dataset of Intended Sarcasm, contained 4,484 English tweets: 777 sarcastic tweets and 3,707 non-sarcastic tweets. It received 1,236 survey responses, with each response containing four tweets labeled by the author.
The contributor profile was concentrated geographically and by age. Fifty-two percent of contributors were from the UK, 32% were from the US, and the two countries together accounted for 84%. Forty-eight percent of contributors were female, and over 68% were under 35 years old. These measurements describe contributors, not necessarily the authors of every tweet or the wider Twitter population.
The text measurements are short-form by design. The average iSarcasm tweet length was around 20 words. The average explanation length was 21 words, while the average rephrase length was 14 words. Over 46% of the tweets were posted in 2019.
The GitHub iSarcasm repository describes the main split as 80% of examples for training and 20% for testing. That division is a dataset-use detail rather than a claim about Twitter activity. It tells readers how the benchmark separated examples for model development and evaluation.
| iSarcasm measure | Reported value |
|---|---|
| English tweets | 4,484 |
| Sarcastic tweets | 777 |
| Non-sarcastic tweets | 3,707 |
| Survey responses | 1,236 |
| Average tweet length | Around 20 words |
| Average explanation length | 21 words |
| Average rephrase length | 14 words |
| Main training/testing split | 80% / 20% |
These figures make sarcasm more concrete as a writing problem. A short tweet may require an explanation longer than the tweet itself, while a rephrase may be shorter than both. The numbers do not prove that sarcasm always needs extra context, but they show the annotation structure used by this corpus.
Personalized sarcasm datasets
Personalized Intended and Perceived Sarcasm Detection on Twitter separates two perspectives that are often merged: intended sarcasm and perceived sarcasm. In the personalized Twitter sarcasm dataset, 75.2% was intended sarcasm and 24.8% was perceived sarcasm. The dataset contained 35K labeled tweets, including 19K non-sarcastic tweets, 11K eliciting tweets, and 4K oblivious tweets.
The collection was built from 33K different authors and included 89M historical tweets for those authors. The number of historical tweets per author ranged from 1 to 500, with an average historical tweet count of 471.46. This historical context is why the dataset is described as personalized: the research did not treat every tweet as isolated from its author.
The author counts were also reported by perspective. There were 15,884 sarcastic authors, including 12,245 intended-sarcasm authors and 3,686 perceived-sarcasm authors. Forty-seven authors appeared in both sarcasm perspectives. There were 17,340 non-sarcastic authors, and 99 authors appeared in both non-sarcastic groups.
Conversation structure supplied another set of measurements. Eighty percent of sarcastic threads were four tweets or fewer. Within the reported sarcastic-thread patterns, second-person AB sequences accounted for 82%, while first-person ABAC sequences accounted for 27% of that class. These percentages refer to the study’s defined sequence classes, so they should not be generalized to every Twitter conversation.
Model performance and benchmark results
The personalized sarcasm experiments used several evaluation setups. On the 34,938-example sarcasm-detection dataset, GNN reached 82.0 F1 and 82.2 accuracy. SBERT reached 74.4 F1 and 74.5 accuracy, A-SBERT reached 80.1 F1 and 80.1 accuracy, the author-attribution model reached 79.3 F1 and 79.3 accuracy, and the priming model reached 77.5 F1 and 77.7 accuracy.
When eliciting and oblivious context was included, the best sarcasm-detection F1 rose to 85.6 for the author-attribution model. SBERT reached 84.9 F1 and 84.9 accuracy with that context. The change highlights the role of surrounding information in personalized sarcasm detection, although the two setups should be kept separate when comparing scores.
For perspective classification, the dataset size was 16,278 examples. GNN reached 72.2 F1 and 80.8 accuracy. AA reached 71.3 F1 and 82.2 accuracy. SentenceBERT reached 68.5 F1 and 79.2 accuracy, Priming reached 70.9 F1 and 79.8 accuracy, and A-SBERT reached 70.6 F1 and 79.2 accuracy.
SemEval 2021 Task 7 produced another group of benchmark figures. Team_KGP reported 0.90 F1 on humor detection, 0.53 F1 on the third subtask, 0.57 RMSE on the second subtask, and 0.58 RMSE on the fourth subtask. UPB’s ensemble achieved 95.66% F1 and 94.70% accuracy on Task 1a, plus 0.6200 RMSE on Task 1b and 0.5318 RMSE on Task 2.
HumorHunter’s evaluation-phase results were 0.9480 F-score on Task 1a, 0.5510 RMSE on Task 1b, 0.4764 F-score on Task 1c, and 0.4230 RMSE on Task 2. F1, F-score, accuracy, and RMSE are not interchangeable. In particular, lower RMSE indicates a smaller prediction error under that task’s definition, while F1 balances precision and recall. The reported values are therefore best read as task-specific benchmark results, not as one universal ranking of humor ability.