Skip to content
Concordia TesseraeConcordia Tesserae

Finaverse Documentation

Welcome

Section titled “Welcome”

Welcome to the Finaverse documentation! Finaverse turns financial news into something you can analyse. It scrapes articles from several sources, uses an LLM to pull out the companies, funds and other securities each article is about (with a sentiment score for each), stores everything in PostgreSQL, and gives you a dashboard to explore it — including a full market picture for any stock in your data, pulled live from Yahoo Finance.

What is Finaverse?

Section titled “What is Finaverse?”

Finaverse combines:

  • A scraping pipeline: Playwright for the sites that need a browser, an RSS feed for the one that will not allow it, and one config object per source
  • LLM entity extraction: which companies an article covers, how central each is to it, and what its sentiment is — with mentions that are only cited in passing thrown away before they cost a ticker lookup
  • Live market data: prices, fundamentals, technicals, peers and earnings from Yahoo Finance, next to your news coverage of the same stock
  • A transparent buy signal: eight scored factors, three time horizons, fixed weights, and an LLM write-up that is checked against the numbers before you see it
  • A signal history: once a day, every stock’s signal and the data behind it are logged, so the scorecard can eventually be tested against what prices actually did
  • A small, deliberate stack: FastAPI, PostgreSQL, Ollama, and plain JavaScript with Chart.js on the front end

Key Features

Section titled “Key Features”

📰 News Crawler

Section titled “📰 News Crawler”

Tick sources, click Scrape Articles, and article cards land live as the job runs. Open a card for the article’s summary and every entity found in it, each with a sentiment chip. Morningstar and Yahoo Finance are scraped with headless Chromium; MarketWatch is read from its public RSS feed.

📊 Analytics

Section titled “📊 Analytics”

Five charts over every stored mention — sentiment trend, net sentiment, entity mentions, quote types, equity sectors — all driven by one shared filter. Every bar and point is a link to the exact rows behind it.

📈 Quote Analysis

Section titled “📈 Quote Analysis”

Pick one stock and see it from two angles at once: what the market says (price history, valuation, profitability, growth, technicals, peers, earnings) and what your news coverage says (sentiment trend, recent mentions, an overlay on the price chart). Each half loads and fails on its own.

🎯 Buy Signal & Ranking

Section titled “🎯 Buy Signal & Ranking”

A Buy / Hold / Avoid reading at short, medium and long horizons, built from a scorecard you can inspect down to each breakpoint. Rank every stock of a quote type against the others, and read the rank as much as the label.

🗂️ Data Explorer

Section titled “🗂️ Data Explorer”

The raw tables behind every chart — with search, sorting, pagination and drill-downs — through a whitelist so nothing from a request ever reaches SQL as an identifier.

🕒 Signal History

Section titled “🕒 Signal History”

A scheduler thread inside the app snapshots every signal and the daily prices once a weekday evening. Rows are stamped with a fingerprint of the scoring rules, so history made under different rules is never silently mixed.

Start here

Section titled “Start here”

Architecture Map — pick something a user does and watch it trace through the codebase: which file does what at which point, and what else depends on that file. It is the fastest way in if you are new to the project, and the quickest way to see the blast radius of a change if you are not.

Then, for depth:

  1. Backend API — every route, and how the background jobs work
  2. Scraping Pipeline — from a news page to stored entities
  3. Database — the twelve tables, and what survives a wipe
  4. Quote Analysis and Signal Engine — the market half of the app

Technology Stack

Section titled “Technology Stack”

Backend

Section titled “Backend”
  • FastAPI + uvicorn: routes, request validation, one worker process
  • PostgreSQL 16 via psycopg2: a real connection pool, JSONB for the signal history
  • Playwright: headless Chromium for scraping
  • yfinance: Yahoo Finance data (unofficial, cached 5 minutes)
  • pandas: technicals computed from price history

Frontend

Section titled “Frontend”
  • Vanilla JavaScript: no framework, no build step
  • Chart.js 4.4.7: every chart, themed from CSS custom properties
  • marked + DOMPurify: Markdown rendering for LLM-written text

AI

Section titled “AI”
  • Ollama: mistral-small locally, or gpt-oss:120b-cloud through Ollama’s cloud
  • A single wrapper: every call is timed, retried and JSON-cleaned in one place

Project Structure

Section titled “Project Structure”
Finaverse/
├── app.py                    # FastAPI routes, lifespan, request models
├── jobs.py                   # In-memory background-job runner
├── data_processing.py        # Scrapers, per-article pipeline, relevance backfill
├── genai_processing.py       # Every LLM call
├── prompts.py                # Every prompt template
├── entity_classifier.py      # Ticker lookup and match checks
├── quote_analysis.py         # Yahoo Finance: metrics, technicals, peers, earnings
├── signal_engine.py          # The scorecard (pure, tested)
├── signal_service.py         # Scorecard + Yahoo + database + LLM write-up + ranking
├── signal_history.py         # The daily snapshot (also a CLI)
├── snapshot_utils.py         # Signal → stored row (pure, tested)
├── scheduler.py              # The in-app scheduler thread
├── schedule_rules.py         # "Is a run due?" as pure functions
├── db/
│   ├── db_init.py            # Creates the database, pool and tables
│   ├── db_config.py          # DatabaseHandler + one class per table
│   ├── queries.py            # The SQL the routes run
│   ├── table_views.py        # Whitelist behind the Data page
│   ├── entity_keys.py        # Name normalisation for de-duplication
│   ├── prune_irrelevant.py   # One-off cleanup CLI
│   └── merge_duplicate_entities.py  # One-off cleanup CLI
├── templates/index.html      # The single-page shell
├── static/                   # script.js, charts.js, css/style.css
├── tests/                    # Scorecard, snapshot rows, schedule rules
├── docs/                     # signal-engine.md, development-notes.md
├── Dockerfile
└── docker-compose.yml

Next Steps

Section titled “Next Steps”