Data_Visualization_ETL_Pipeline

Visualizing country data with an ETL(Exract Transform Load) pipeline using DAO(Data Access Object) pattern. -UOI Project

Prerequisite 👋

• MySQL - Download here

• MySQL Workbench - Download here

• Jupyter Notebook - Download here

• The Dataset that we will be using is being downloaded from TheWorldBank and is the 'countries.csv'

Step 1 : Clean Dataset & Create Database 📝

1. We use the CleanData.ipynb to clean the countries_data.csv dataset and extract only the NY,SPand SE families of Code Indicators. We store the data in 3 csv files countries.csv , indicators.csv and year_value.csv.
2. We create a schema in MySQL Workbench and load the data (using the LOAD DATA INFILE statement).In MySQL database we will have 3 tables:

• country(id, Name, Code)
• indicator(id, Name, Code)
• indicates(id_country, id_ind, year, value)

Step 2 : Create packages - Start building the app 📦

We created the structure of the app and then we did the implementation.

• View package: create the user interface with fxml files
• Controller package : take inputs from fxml files and do some process if is needed
• DAO package : create objects that interact with MYSQL database

SQL-Schema