Data Scientist Roadmap
From Python, SQL and statistics to machine learning, experimentation, causal inference and dashboards, built around what data science job postings ask for. Free resources in English and Hindi.
Data types, lists and dicts, comprehensions, functions, modules, file I/O and reading CSV/JSON
Jupyter/JupyterLab, Google Colab, cells and kernels, magics, virtual environments
ndarrays, dtypes, indexing and slicing, broadcasting, vectorization, aggregations, random
Series and DataFrames, loc/iloc, filtering, groupby, merge/join, pivot and reshape, datetime, apply
Lazy vs eager API, expressions, group_by, joins, and when to use Polars over pandas
R syntax, tibbles, dplyr verbs (filter, select, mutate, summarise), ggplot2, the tidy workflow
Personal Finance Analyzer
beginnerLoad a year of bank or UPI transaction CSVs, clean and categorize them with pandas, and report monthly spend by category in a Jupyter notebook.
- Python
- pandas
- Jupyter
NumPy-only Statistics Toolkit
beginnerImplement mean, variance, percentiles, z-scores and a vectorized moving average in pure NumPy, then benchmark them against Python loops and pandas.
- Python
- NumPy
pandas vs Polars Benchmark
intermediateRun the same groupby/join/filter pipeline on a 5M+ row public dataset (e.g. NYC taxi trips) in pandas and Polars (lazy) and write up timings and memory use.
- Python
- pandas
- Polars
- Parquet
SELECT, WHERE, ORDER BY, LIMIT, DISTINCT, NULL handling, CASE WHEN, basic functions
INNER/LEFT/FULL/self joins, GROUP BY, HAVING, aggregate functions, subqueries, set operations
ROW_NUMBER, RANK/DENSE_RANK, LAG/LEAD, running totals, PARTITION BY, WITH clauses, recursive CTEs
Indexes, EXPLAIN/EXPLAIN ANALYZE, sargable predicates, avoiding SELECT *, join cost
Hands-on analytics problems: funnels, retention, top-N per group, date math, rolling metrics
E-commerce Sales Analysis in SQL
beginnerLoad a public e-commerce dataset (e.g. Olist) into PostgreSQL or SQLite and answer 15 business questions using joins and aggregations: revenue by month, top categories, repeat buyers.
- SQL
- PostgreSQL
- SQLite
Cohort Retention & Funnel Queries
intermediateWrite CTE- and window-function-based queries for monthly cohort retention, conversion funnels, and 7-day rolling active users, then chart the output in a notebook.
- SQL
- PostgreSQL
- Window functions
- Python
- +1
Query Tuning Case Study
advancedGenerate a multi-million-row table, profile slow analytics queries with EXPLAIN ANALYZE, add indexes or rewrite queries, and document the before/after timings.
- PostgreSQL
- EXPLAIN
- Indexes
Mean, median, mode, variance, standard deviation, percentiles, IQR, skewness, correlation
Rules of probability, conditional probability, random variables, expectation, Bernoulli/binomial/Poisson/normal distributions
Sampling methods and bias, sampling distributions, Central Limit Theorem, standard error, confidence intervals, bootstrap
- CRSKhan Academy - Sampling Distributions
- VID3Blue1Brown - But what is the Central Limit Theorem?
- VIDStatQuest with Josh Starmer - Confidence Intervals, Clearly Explained
- BOOKCetinkaya-Rundel & Hardin - Introduction to Modern Statistics
- PLAYLISTcodebasics Hindi - Mathematics, Statistics for Data Science and ML in HindiHINDI
Null and alternative hypotheses, p-values, t-tests, chi-square, Type I/II errors, power, multiple testing, A/B test design
Bayes' theorem, priors and posteriors, Bayesian updating, Bayesian vs frequentist intuition
Simple and multiple linear regression, OLS, coefficients and interpretation, residuals, R-squared, assumptions
Simulating the Central Limit Theorem
beginnerSimulate sampling from skewed distributions (exponential, Poisson) and visualize how sample means converge to normal; add bootstrap confidence intervals.
- Python
- NumPy
- Matplotlib
A/B Test Analysis Report
intermediateAnalyze a public A/B testing dataset (e.g. Kaggle cookie-cats or marketing A/B): check sample-ratio mismatch, run a proportion z-test and a bootstrap, compute power, and write a stakeholder recommendation.
- Python
- SciPy
- statsmodels
- pandas
Explaining House Prices with Regression
intermediateFit and interpret an OLS model on a housing dataset with statsmodels: coefficients, confidence intervals, residual diagnostics, multicollinearity (VIF).
- Python
- statsmodels
- pandas
- Seaborn
Tidy data principles, type fixing, string cleanup, duplicates, parsing dates, reshaping (melt/pivot)
MCAR/MAR/MNAR, dropping vs imputation, simple and iterative imputers, IQR and z-score outlier detection
Univariate and bivariate analysis, distributions, correlations, segment comparisons, forming hypotheses
- VIDfreeCodeCamp.org - Data Analysis with Python: Full Course (NumPy, Pandas, Matplotlib, Seaborn)
- BOOKRoger D. Peng & Elizabeth Matsui - The Art of Data Science
- BOOKHadley Wickham et al. - R for Data Science (2e): Exploratory Data Analysis
- REPOMicrosoft - Data Science for Beginners (lessons 7-8: Python & Data Preparation)
- PLAYLISTCampusX - 100 Days of Machine Learning (EDA & data-gathering days)HINDI
Figure/axes anatomy, line/bar/hist/scatter/box plots, subplots, statistical plots, styling
Plotly Express, hover and facet charts, maps, and simple Dash apps
Choosing the right chart, color, perception, avoiding misleading plots, storytelling for stakeholders
Messy Dataset Cleanup
beginnerTake a deliberately messy public dataset (e.g. Kaggle Netflix titles or Zomato restaurants), fix types, duplicates, missing values and outliers, and document every decision in a reproducible notebook.
- Python
- pandas
- Jupyter
End-to-end EDA Report
intermediateRun a full EDA on a Kaggle dataset (e.g. IPL matches, Indian startup funding): univariate/bivariate analysis, key insights, and a one-page summary written for a non-technical stakeholder.
- Python
- pandas
- Seaborn
- Matplotlib
Interactive Insights Dashboard
advancedBuild a Plotly Dash dashboard over an Our World in Data dataset with filters, linked charts and a written narrative, then deploy it for free.
- Python
- Plotly
- Dash
- pandas
Linear and logistic regression, regularization (ridge/lasso), k-NN, the bias-variance trade-off, and framing business problems as prediction tasks.
Decision trees, bagging and random forests, gradient boosting, XGBoost/LightGBM hyperparameters, and why boosted trees dominate tabular data.
k-means, hierarchical clustering and DBSCAN, choosing k, PCA for dimensionality reduction, and customer segmentation use cases.
Train/validation/test splits, k-fold cross-validation, precision/recall/F1, ROC-AUC, regression metrics, overfitting, and picking metrics that match business cost.
Encoding, scaling, imputation, target/date features, ColumnTransformer and Pipeline, and avoiding data leakage.
Permutation importance, partial dependence, SHAP values for global and local explanations, and explaining models to stakeholders.
House Price Regression Baseline
beginnerPredict house prices on a Kaggle dataset, comparing linear regression, ridge/lasso and a random forest with cross-validated RMSE.
- Python
- pandas
- scikit-learn
Customer Segmentation
beginnerCluster e-commerce customers on RFM features with k-means and PCA, then write a one-page persona summary for each segment.
- Python
- pandas
- scikit-learn
- matplotlib
Explainable Churn Model
intermediateBuild a leakage-safe ColumnTransformer + LightGBM churn pipeline, tune it with CV, pick a threshold from business costs, and explain drivers with SHAP.
- Python
- scikit-learn
- LightGBM
- SHAP
Trend, seasonality and cycles, STL decomposition, stationarity and differencing, autocorrelation (ACF/PACF), and lag features.
Simple benchmarks (naive, seasonal naive), ETS/Holt-Winters, ARIMA and SARIMA, and automatic model selection.
Prophet for business series with holidays, lag/rolling features with gradient boosting, multi-series (global) models, and direct vs recursive strategies.
Scale-free error metrics (MAE, RMSE, MAPE, MASE), rolling-origin cross-validation, prediction intervals, and beating naive baselines.
Website Traffic Forecast with Prophet
beginnerForecast daily page views with holiday effects, and compare against a seasonal-naive baseline using Prophet's cross-validation.
- Python
- Prophet
- pandas
Retail Demand Forecasting Benchmark
intermediateBacktest ETS/ARIMA (StatsForecast) against a LightGBM global model on multi-store sales, reporting MASE and prediction-interval coverage.
- Python
- StatsForecast
- MLForecast
- LightGBM
Hierarchical Inventory Forecasting
advancedForecast SKU/store/region demand, reconcile across levels, and turn forecasts into reorder recommendations with a simple cost simulation.
- Python
- Nixtla
- pandas
- Plotly
Hypotheses, randomization units, control vs treatment, why randomization gives causal estimates, and the end-to-end experiment lifecycle.
Type I/II errors, minimum detectable effect, power analysis and sample size, test duration, and p-values vs confidence intervals.
Choosing an overall evaluation criterion, guardrail and sensitivity metrics, ratio metrics, and common metric interpretation pitfalls.
Peeking and optional stopping, sequential testing, novelty and primacy effects, sample ratio mismatch, multiple comparisons, and trustworthiness checklists.
- ARTEvan Miller - How Not To Run an A/B Test
- ARTEvan Miller - Simple Sequential A/B Testing
- PPRCrook, Frasca, Kohavi, Longbotham - Seven Pitfalls to Avoid when Running Controlled Experiments on the Web
- PPRFabijan et al. (Microsoft) - Three Key Checklists and Remedies for Trustworthy Analysis of Online Controlled Experiments at Scale
Using pre-experiment covariates (CUPED/regression adjustment) to shrink variance, run shorter tests, and detect smaller effects.
Mobile Game A/B Test Analysis
beginnerAnalyze a public gate-placement A/B test (e.g. Kaggle Cookie Cats): run SRM checks, compute retention lift with bootstrap CIs, and write a ship/no-ship memo.
- Python
- pandas
- SciPy
- statsmodels
Experiment Planning Calculator
intermediateBuild a Streamlit app that computes sample size, test duration and MDE for conversion and continuous metrics, with a simulation that shows how peeking inflates false positives.
- Python
- statsmodels
- NumPy
- Streamlit
Experiment Analysis Toolkit
advancedWrite a reusable library that runs SRM detection, CUPED variance reduction, delta-method ratio metrics and sequential testing, and validate it on simulated A/A and A/B data.
- Python
- NumPy
- pandas
- statsmodels
- +1
Correlation vs causation, potential outcomes and treatment effects (ATE/ATT), confounding, selection bias, and causal graphs.
Unconfoundedness, propensity score estimation, matching and inverse probability weighting, doubly robust estimation, and checking balance and overlap.
Quasi-experiments when you can't randomize: the parallel trends assumption, two-way fixed effects, event studies, and synthetic control for geo or market-level launches.
Conditional average treatment effects (CATE), meta-learners (S/T/X-learner), uplift models for targeting, and evaluating them with uplift/Qini curves.
Policy Impact with Difference-in-Differences
intermediateEstimate the effect of a city- or state-level policy (e.g. a minimum wage change) with DiD, plot pre-trends, and run placebo tests.
- Python
- pandas
- statsmodels
- linearmodels
Observational Program Evaluation
intermediateEstimate a job-training program's effect on the LaLonde data with propensity score matching and IPW, check covariate balance, and compare against the experimental benchmark.
- Python
- scikit-learn
- DoWhy
- pandas
Uplift-Based Marketing Targeting
advancedTrain T-/X-learner uplift models on a randomized marketing dataset, evaluate them with Qini curves, and compute the profit of targeting the top uplift deciles.
- Python
- CausalML
- LightGBM
- pandas
Neurons, layers and activations, backpropagation intuition, training loops in PyTorch or Keras, and when deep learning beats boosted trees (and when it doesn't).
Tokenization, bag-of-words/TF-IDF baselines, word and sentence embeddings, transformers at a high level, and using pretrained models from Hugging Face.
Calling LLM APIs, prompt design for extraction and labeling, zero/few-shot text classification, embedding-based classifiers, and validating LLM labels against a human-labeled sample.
- CRSDeepLearning.AI (Isa Fulford & Andrew Ng) - ChatGPT Prompt Engineering for Developers
- DOCOpenAI Cookbook - Classification using embeddings
- DOCAnthropic Docs - Ticket routing (LLM text classification guide)
- DOCHugging Face Transformers - Text classification task guide
- PLAYLISTKrish Naik Hindi - Complete Generative AI And Agentic AI PlaylistHINDI
Vector similarity search, chunking and retrieval, retrieval-augmented generation over documents or data dictionaries, and where data science hands off to AI engineering.
Support Ticket Classifier Shoot-out
intermediateClassify support tickets three ways (TF-IDF + logistic regression, sentence embeddings + classifier, zero-shot LLM) and compare accuracy, cost and latency on a labeled hold-out set.
- Python
- scikit-learn
- sentence-transformers
- LLM API
Customer Review Theme Mining
intermediateEmbed product reviews, cluster them into themes, use an LLM to name and summarize each cluster, and track theme sentiment over time in a dashboard.
- Python
- sentence-transformers
- scikit-learn
- LLM API
- +1
Ask-Your-Data-Dictionary RAG Assistant
advancedBuild a RAG assistant over a company's metric definitions and analysis docs that answers questions with citations, and evaluate retrieval quality on a small test set.
- Python
- FAISS
- LangChain
- LLM API
Cleaning data in spreadsheets, lookups (XLOOKUP), conditional aggregation, PivotTables and PivotCharts, and simple interactive dashboards.
- DOCMicrosoft Support - Excel Help & Learning (training)
- DOCMicrosoft Support - Overview of PivotTables and PivotCharts
- DOCGoogle Docs Editors Help - Create & use pivot tables
- VIDLuke Barousse - Excel for Data Analytics: Full Course for Beginners
- VIDRishabh Mishra - Full Project in Excel with Interactive DashboardHINDI
Power Query for getting and transforming data, star-schema data modeling, DAX measures, report visuals and publishing to the Power BI service.
- CRSMicrosoft Learn - Get Started with Microsoft Data Analytics (Power BI)
- CRSMicrosoft Learn - Model Data with Power BI
- DOCMicrosoft Learn - DAX Reference
- VIDLuke Barousse - Power BI for Data Analytics: Full Course for Beginners
- VIDRishabh Mishra - Power BI Tutorial for Beginners (Dashboard Project in Hindi)HINDI
Connecting data, dimensions vs measures, calculated fields, filters and parameters, dashboards and stories published to Tableau Public.
Free Google dashboarding tool: connecting Sheets and BigQuery data sources, calculated fields, controls and filters, and sharing reports.
Choosing the right chart, decluttering, using preattentive attributes for focus, building a narrative from question to insight to recommendation, and presenting to stakeholders.
Sales Insights Dashboard
beginnerModel a multi-table sales dataset into a star schema, write DAX measures for revenue, margin and YoY growth, and publish an interactive dashboard with a one-page executive summary.
- Power BI
- Power Query
- DAX
Tableau Public Data Story
beginnerPick a public dataset (for example, Indian census or World Bank data), build 3-4 linked dashboards with filters and parameters, and publish a Tableau Public story that answers one clear business question.
- Tableau Public
- Excel
Self-Updating KPI Report
intermediatePull data from Google Sheets or the BigQuery public datasets into a Looker Studio report with calculated KPIs, date controls and scheduled email delivery, then write a short insights memo.
- Looker Studio
- Google Sheets
- BigQuery
OLTP vs OLAP, data warehouse vs data lake vs lakehouse, star and snowflake schemas, fact and dimension tables, and partitioning and clustering.
Extract-transform-load vs extract-load-transform, batch ingestion, idempotent pipelines, DAGs, scheduling and retries with orchestrators such as Airflow or Kestra.
SQL models, sources and refs, materializations, tests, documentation and lineage, and transforming raw warehouse tables into analysis-ready marts.
Distributed computing basics, Spark DataFrames and Spark SQL, lazy evaluation, transformations vs actions, joins, shuffles and partitioning.
Querying BigQuery in the free sandbox, Azure core data concepts and analytics services (DP-900 level, common in Indian enterprise roles), and notebooks on Databricks Free Edition.
Git basics (commit, branch, merge, pull requests), .gitignore for data and notebooks, and versioning datasets and models with DVC.
ELT Pipeline: Raw Data to Analytics Mart
intermediateIngest NYC taxi (or another public) data into BigQuery, build staging and mart models with dbt including tests and docs, schedule the pipeline, and point a dashboard at the mart.
- Python
- BigQuery
- dbt
- Airflow or Kestra
- +1
PySpark Batch Analysis
intermediateProcess a multi-GB public dataset with PySpark on Databricks Free Edition: clean, join and aggregate it, compare DataFrame and Spark SQL approaches, and write partitioned Parquet outputs.
- PySpark
- Spark SQL
- Databricks
- Parquet
Reproducible Data Project Repo
beginnerSet up a Git repo for an analysis project with DVC-tracked raw data, a reproducible pipeline (dvc repro), branch-based experiments and a clear README.
- Git
- GitHub
- DVC
- Python
Turning notebooks into web apps with widgets, caching and state, Gradio interfaces for model demos, and free hosting on Streamlit Community Cloud or Hugging Face Spaces.
REST API basics, Pydantic request and response schemas, loading a trained model at startup, online vs batch inference, and testing endpoints.
Images vs containers, writing a Dockerfile for a Python or ML app, ports and volumes, Docker Compose, and pushing images to a registry.
Logging parameters, metrics and artifacts, comparing runs, autologging, and registering, versioning and loading models from the MLflow Model Registry.
Data drift vs concept drift, monitoring prediction and data-quality metrics, drift reports with Evidently, and deciding when to retrain.
Public ML Demo App
beginnerTrain a tabular or text model and ship it as a Streamlit or Gradio app on Hugging Face Spaces or Streamlit Community Cloud, with input validation and an explanation of each prediction.
- Python
- scikit-learn
- Streamlit
- Gradio
- +1
Containerized Prediction API
intermediateWrap a model in a FastAPI service with Pydantic schemas and tests, containerize it with Docker, deploy it to a free cloud tier, and document the endpoints in the README.
- FastAPI
- Docker
- Pydantic
- pytest
- +1
Tracked and Monitored Model Lifecycle
advancedLog experiments and register the best model in MLflow, serve it, simulate weekly production batches, and generate Evidently drift reports that trigger a retraining decision.
- MLflow
- Evidently
- FastAPI
- Docker
- +1
North Star and input metrics, acquisition, activation, retention and churn, funnels and cohorts, guardrail metrics, the HEART framework, and A/B test basics.
Structuring open-ended product questions: clarifying the goal, defining success metrics, diagnosing a metric drop, sizing impact, and proposing experiments.
Two deployed projects on public GitHub count for more than certificates: ship an app or API and write a README covering the problem, data, approach, results and a live link. Then target internships, which about 60% of Indian fresher postings prefer, and tailor your resume to each role.
Reading competition rules and metrics, building a validation setup that matches the leaderboard, feature engineering, learning from public notebooks, and writing up solutions.
Metric Drop Investigation
intermediateUsing an e-commerce or app event dataset, define a North Star metric and its input metrics, simulate a 10% drop, and run a structured root-cause analysis in SQL with a written recommendation.
- SQL
- BigQuery
- Python
- Looker Studio
Deployed Portfolio of Two End-to-End Projects
intermediatePublish two public GitHub repos, each with a deployed app or API (Streamlit, Hugging Face Spaces or FastAPI on a free tier) and a README covering the problem, data, method, results and a live demo link, collected on a GitHub Pages portfolio site.
- GitHub
- GitHub Pages
- Streamlit
- FastAPI
- +1
Kaggle Competition Write-Up
advancedEnter an active or Getting Started Kaggle competition, build a solid cross-validation baseline, iterate on features and models, and publish a public notebook and blog-style write-up of what worked.
- Python
- pandas
- scikit-learn
- LightGBM
- +1