📌 Project Overview

https://github.com/nevenjer/companiesmarketcap-playwright/tree/main/sql

Project 18.1 extends the CompaniesMarketCap automation pipeline by integrating the collected company and historical ranking data with Microsoft SQL Server.

The objective is to transform the scraped JSON datasets into a structured relational database and apply SQL-based data validation to ensure that the stored data is complete, consistent, and suitable for future financial data analysis.

This project is the database layer between the web automation pipeline and the future R Market Data Pipeline (Project 18.2).

🎯 Project Objective

Build a reliable SQL Server data storage and validation layer for the CompaniesMarketCap dataset.

The project focuses on:

• Designing relational database tables
• Creating reusable SQL Server setup scripts
• Importing JSON data into SQL Server
• Maintaining a company master dataset
• Maintaining historical company ranking data
• Detecting duplicate records
• Detecting NULL values
• Validating ranking coverage
• Cross-validating datasets
• Preparing structured data for future R-based financial analysis