Data · dataset · 2025
A Bilingual Malay-English Social Media Dataset for Binary Hate Speech Detection
Listed in Teesside University Research Data Repository
This dataset consists of 26,985 bilingual Malay-English social media posts curated for binary hate speech detection tasks.
Description
The data are collected and processed from five publicly available sources: HateXplain (English), HateM (Malay), Toxicity-Small (Malay), Snapshot-Twitter-2022 (Malay), and Supervised-Twitter (Malay). Among them, HateXplain, HateM, and Toxicity-Small contain manually annotated labels, while Snapshot and Supervised-Twitter are pseudo-labeled using a custom pipeline.
Each entry includes cleaned social media text, a binary hate speech label (0 = non-hate, 1 = hate), language code (en for English, ms for Malay), and the original source. The dataset is balanced across classes, with 14,642 non-hate and 12,343 hate-labeled entries. Language distribution includes 13,609 English and 13,376 Malay-language texts.
Read the rest (2 more)
Preprocessing is conducted separately for each language. Malay-language texts underwent spelling correction, slang normalisation, placeholder standardisation (<user>, <number>), and translation filtering using Malaya NLP tools and custom-built dictionary (malayslangdict.py in slang_dictionary_custom folder). English texts are processed using Ekphrasis to handle social media-specific patterns, emojis, hashtags, and slang.
Low-quality or duplicate entries are removed to ensure the data quality. The dataset is stored in CSV format (UTF-8) and is suitable for training and evaluating multilingual and low-resource hate speech detection models.
Links
Where it is published
- DOI doi.org/10.17632/mgv2n2vcb9.3 ↗
DOI / persistent id · from researchdata tees ac uk
Catalogue records · 1
- OAI-PMH record data.mendeley.com/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Adata… ↗
metadata API · from researchdata tees ac uk
Topics
- From keywords
- Computer Science & AI · Earth & Environmental Science · Humanities · Natural language processing
Provenance · 1 source records, 11 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Teesside University Research Data Repository | oai:data.mendeley.com/mgv2n2vcb9.3 | 15 h ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| concepts[field].anzsrc:field:460208 | mapping · researchdata tees ac uk | vocabulary-mapper@1.0.0 | keywords['Natural Language Processing'] |
| concepts[field].local:field:computer-science-ai | mapping · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| concepts[field].local:field:humanities | mapping · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| concepts[modality].local:modality:audio | enrichment · researchdata tees ac uk | keyword-concept-rules@1.0.0 | title+description (65%) |
| concepts[modality].local:modality:text | enrichment · researchdata tees ac uk | keyword-concept-rules@1.0.0 | title+description (75%) |
| description | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | /metadata/dc/description |
| license | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | /metadata/dc/rights |
| publication_date | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| title | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | /metadata/dc/title |