1. Project Overview

https://github.com/nevenjer/companiesmarketcap-playwright/blob/main/docs/market-price-ingestion.md

Project 18.2 extends Project 18.1 from company and ranking data collection into a historical market price data pipeline.

The main objective is to build a reliable pipeline that can:

• Retrieve historical market price data from Yahoo Finance
• Map market data to internal company_id
• Store daily OHLCV data in SQL Server
• Prevent duplicate business records
• Support incremental ingestion
• Update existing records and insert new records
• Process multiple companies
• Handle transaction rollback safely
• Validate database integrity
• Identify failed symbols without corrupting successful data
• Keep database credentials outside source code
• Prepare the project for future large-scale market data processing

2. Project Journey

The project evolved through several stages.