Back to home2025–26
Detecting Portuguese fake news
Five classifiers trained on 63,000 European Portuguese news articles, and a close look at why they decide what they decide.
- Who it’s for
- Fact-checkers and newsrooms working in European Portuguese, where off-the-shelf tools are thin.
- What it changed
- Showed that a small network on word counts gets within 0.02 F1 of a fine-tuned BERT, at a fraction of the compute. Second place, and a published paper.
- Where
- NTT Data & Técnico Merit Prize 2025–26
- Outcome
- Second place, paper on Zenodo
- Team
- Two people, with Daniel Borges
- Built with
- Python, scikit-learn, PyTorch, LIME, TF-IDF, K-Means, PCA
Paste a Portuguese news article and see what the model thinks
This runs one of the project’s trained networks directly in your browser: 2.6 MB of weights, and your text never leaves this page. It scores 0.923 F1 on the held-out test set. It reads vocabulary, not facts, so treat its verdict as a guess about style.
Try an article from the test set
Why: each word is shaded by how much the verdict moves when that word is taken out
pushes towards real pushes towards fake
Model card
- Model
- TF-IDF over 5,000 terms, then an MLP with one hidden layer of 256 ReLU units. 1.28 million weights, shipped as float16 (2.6 MB).
- Trained on
- 50,587 European Portuguese news articles from the FakeNews-PT corpus, each labelled real or fake. Punctuation and stop words removed.
- Score
- 0.923 macro-F1 on 6,324 held-out test articles.
- Runs
- In your browser, in plain TypeScript. No server sees your text.
Meant for A teaching demo of what a text classifier picks up from a news corpus. It is not a fact-checker, and nobody should use it to decide whether a story is true.
Where it fails
It judges style, not truth
The model has never seen a fact. It learned what real and fake articles sound like. A made-up story written like a news-agency wire comes out as real with high confidence, mostly thanks to words like “anunciou” and “Lusa”.
And the reverse
A plainly true sentence in clickbait clothing (“Chocante!”, “ninguém lhe contou”, “veja as imagens”) is called fake with near certainty.
Its world stops at the training data
Words outside its 5,000-term vocabulary are ignored, so new names and events are invisible to it. With only a sentence or two, a handful of known words decide the outcome and the confidence means little.
The problem
Automated fake-news detection is fairly mature for English and much thinner for Portuguese, especially European Portuguese. The challenge, set by NTT Data and Técnico’s Machine Learning course, was to classify articles from the FakeNews-PT corpus as real or fake, and then explain what the models had actually learned.
The corpus has 63,235 articles (50,587 for training, 6,324 each for validation and test), drawn from the fake-news-pt-eu dataset.
What we did
We built one pipeline and ran everything through it four times, once for each way of cleaning the text: with or without punctuation, and with or without Portuguese stop words. Each version fed a TF-IDF vectoriser capped at 5,000 terms (min_df 10, max_df 0.9). We then grid-searched five classifiers with 5-fold cross-validation, optimising macro-F1: a decision tree, Gaussian naive Bayes, L1 and L2 logistic regression, and a multi-layer perceptron.
Keeping punctuation and dropping stop words worked best on average. Removing punctuation cost the models the most, probably because the exclamation marks and quotes that clickbait relies on carry real signal.
Results
| Model | Test F1 (macro) |
|---|---|
| Decision tree | 0.859 |
| Gaussian naive Bayes | 0.852 |
| Logistic regression, L2 | 0.900 |
| Logistic regression, L1 | 0.901 |
| MLP, 256 → 128 | 0.926 |
The MLP reached an ROC AUC of 0.978. For the final presentation we also fine-tuned BERT, which pushed test F1 to 0.948.
What the models learned
L1 regularisation drove 2,898 of the 5,000 word weights to exactly zero, which leaves a short, readable list of the words that separate real from fake. LIME showed the difference between the two model families on individual articles. The linear model leans on the same global words every time, while the MLP weighs words differently depending on context. That makes its explanations more nuanced, and sometimes less intuitive. Permutation importance over 1,000 validation articles gave the global view to compare against.
We also clustered the corpus without labels. K-Means with five clusters found clear topics: the war in Ukraine, public health, domestic politics, the economy, and curiosities and clickbait. Projecting into 2D, fake articles are denser in one corner but overlap real ones everywhere. In other words, misinformation isn’t confined to a few topics; it imitates whatever is in the news.
About the demo
The demo runs the trained network in your browser with no server. It’s a sibling checkpoint from the same experiments: one hidden layer of 256 units, trained on text without punctuation or stop words. It scores 0.923 F1 on the same test set; the headline 0.926 model’s weights aren’t in the public repository. Word highlights come from occlusion: we take each word out, re-run the model, and measure how far the probability moves. It’s a cheaper cousin of the LIME analysis in the paper.