Léonard Noth
ENFRDE
← projects

SAFE

Reading financial news three questions at a time

Financial news, read sentence by sentence: which company is being talked about, when the event happens, and how it feels. Three separate questions.

Role
Sole author
Scope
Research design · Dataset generation · Model training · Backend services · Interface
Built with
Python, PyTorch, BERT, scikit-learn, spaCy, SUTime, Flask, Vue 3, Docker, GPT-4
Team
Solo
Timeline
Master's thesis, defended 2024, graded 6/6
Outcome
Graded 6 out of 6. Three separable questions answered per sentence fragment, aggregated per company and per time window, with named emotion pairs instead of a forced single winner.

The problem

Off-the-shelf financial sentiment tells you a headline is negative. It rarely tells you negative about whom, or whether the thing described has already happened. A single article can be optimistic about one company and grim about another, and can mix an event from last year with a forecast for next quarter. Treating that as one score throws away most of the signal.

My role

Master's thesis, sole author, supervised by Dr Sandy Ingram. Everything from the dataset generation tooling to the six services and the interface.

Read as
The essentials in two minutes

Impact

99.6% val. accuracy
Temporal classifier
98.8% val. accuracy
Sentiment classifier
96% on 50 hand-labelled samples
Sentence segmentation
6,501 labelled samples
Generated corpus
6 / 6
Grade

The system design

  1. Generate the dataset

    No public dataset existed for sentence-level, finance-specific temporality, so the first deliverable is a tool that builds one.

    • A TOML file describes the classes, the domain and a handful of human-written seed sentences
    • GPT-4 generates and augments from there: 2,501 temporal rows across five classes, 4,000 sentiment rows across eight
  2. Prove the classes are separable

    Before any training, every sample is embedded by three independent encoders and the classes are tested for statistical separation.

    • This is the step that caught fear and sadness overlapping under one encoder
    • A weakness worth knowing before training, not after
  3. Segment and bind to companies

    An article is split into sentences and then into clause-sized fragments, and each fragment is bound to the companies it actually mentions.

    • 96% agreement with hand-segmentation on a 50-sentence benchmark
  4. Two questions at once

    Each fragment goes to two services in parallel: when is this happening, and what does it feel like.

    • Temporality: a rule-based tagger handles explicit dates, a fine-tuned BERT takes over only when no date phrase exists
    • Emotion: eight primary emotions from Plutchik's wheel rather than positive/negative
  5. Aggregate and name the pair

    Results are counted per company and per time window, and when the top two emotions are close the system reports the named pair they form.

    • Anticipation plus joy is optimism, and the composition rule comes from the theory, not from me
    • Eight primary emotions compose into the named dyads rather than being predicted directly
Platform services
  • Six Docker services

    Segmentation, entity binding, temporality, sentiment, the decision maker and the dataset CLI. Separated because their dependencies genuinely conflict

  • SUTime on a JVM

    Rule-based date normalisation; needs Java, which is one reason it lives in its own container

  • Fine-tuned BERT ×2

    One model for temporality, one for emotion, each trained on the generated corpus

  • spaCy

    Sentence splitting, part-of-speech tagging, and the custom entity model that binds fragments to tickers

  • Decision maker

    The single entry point the interface talks to; aggregates and composes the dyads

  • Vue SPA

    Submit an article, read the analysis per company and per time bucket

Key decisions

Predict eight emotions, compose twenty-seven

instead of · Positive / negative / neutral, or predicting all the compound emotions directly

Ternary sentiment can't distinguish fear from anger, which matter differently to a market. Predicting every compound emotion directly means many more classes and worse performance on each. Training on the eight primaries and composing the pairs afterwards keeps the class count low and makes the composition rule defensible: it comes from Plutchik, not from me.

Rules and a model, each doing what it's good at

instead of · One model for all temporal expressions

A fixed set of weights cannot do calendar arithmetic against a moving today, since 2023 is recent in 2024 and ancient in 2030. So explicit dates go to a rule-based tagger, and the model only handles implicit language. The two halves don't overlap by construction: the generated training data was explicitly instructed never to mention specific dates.

Class boundaries as configuration

instead of · Letting the model own what 'recent' means

A day trader's present is hours; a pension fund's is a quarter. If those thresholds live in the weights, serving both audiences means relabelling and retraining. In a config file it is a one-line edit. This is the decision I'd defend hardest.

Build the dataset tool, not just the dataset

instead of · A one-off generation script

Every existing corpus was the wrong granularity, the wrong domain or not public. Making the generator generic, driven by a TOML spec, meant the same tool produced both datasets and can produce the next one. The validation step was built into it rather than bolted on.

In action

The analysis view: per company, per time window, with emotion dyads
The analysis view: per company, per time window, with emotion dyads
Submitting an article for analysis
Submitting an article for analysis
The thesis poster
The thesis poster

Off-the-shelf financial NLP tells you a headline is “negative”. It rarely tells you negative about whom, or whether the thing described already happened. SAFE splits an article into sub-sentence segments, detects the tickers mentioned, and then classifies each segment twice: on a five-class temporal axis (distant past to distant future) and on the eight primary emotions of Plutchik’s wheel. The results are aggregated per company per time bucket, and when two emotions come out close together the system reports the named dyad they form instead of picking a winner. Anticipation plus joy is optimism.

Temporality is deliberately hybrid: Stanford’s SUTime handles explicit dates with rule-based precision, and a fine-tuned BERT takes over only when no date phrase exists. The class boundaries are configuration, not training data, so what counts as “present” can be changed without touching the model.

No dataset existed for either task, so a second deliverable is a CLI that generates and augments labelled data with GPT-4 from a TOML spec and a handful of human-written seed sentences. Before any training, every generated set is validated: each sample is embedded with three independent encoders and pairwise t-tests check that the classes are actually separable. That pass is what caught fear and sadness overlapping under one encoder, a weakness worth knowing before, not after, training.

The honest limitation, stated in the thesis itself: the reported accuracies are validation figures on GPT-4-generated data, with no independent test set. They measure fit to a synthetic distribution, not performance on live market news. And the NER stage only covers NASDAQ listings.

What I learned

  • The most valuable thing I built wasn't a model, it was the thing that let me have a dataset at all.
  • Deciding what a model should not learn is as important as what it should. Explicit dates were deliberately kept away from the network, because they are arithmetic against a moving reference and weights cannot hold that.
  • Predicting eight things well and composing twenty-seven beat predicting twenty-seven things badly.
  • Re-reading my own validation with fresh eyes, the separability test is narrower than it looks: it compares how tightly each class clusters rather than how far apart two classes sit. It still caught a real overlap between fear and sadness, which is what I needed it for, but I'd build it differently now.

What it doesn't do

  • The reported accuracies are validation scores on data generated by the same model that produced the training set. They measure fit to a synthetic distribution, not performance on live market news.
  • A news article isn't objective. One article saying a given period looks negative doesn't make it so. The system measures coverage, not truth.
  • The entity recognition covers NASDAQ listings, and the companies associated with a ticker keep changing; keeping that current would mean continuously relabelling.
  • It deliberately gives no financial advice. I'm not a financial adviser, and the step from sentiment to a position is exactly where that would start.