Context
This project covers the design of a complete data pipeline on Google Cloud Platform, from ingestion to visualization.
Architecture
Public API -> Cloud Function -> Cloud Storage (raw)
-> Dataflow (transformation) -> BigQuery (analytics)
-> Looker Studio (dashboard)
Orchestration: Apache Airflow (scheduling, dependencies, alerting)
Demonstrated skills
- Data ingestion from a REST API
- Structured storage in Cloud Storage
- Transformation and loading into BigQuery
- Task orchestration with Airflow
- Dashboard exposure layer
Tech stack
Python · GCP · BigQuery · Cloud Storage · Dataflow · Airflow · Looker Studio
Results
(To be completed when the project is fully published)