It is a beautiful spring day, and it is two weeks since you have been hired as a new data engineer at Pewlett Hackard. Your first major task is a research project on employees of the corporation from the 1980s and 1990s. All that remain of the database of employees from that period are six CSV files.
In this assignment, you will design the tables to hold data in the CSVs, import the CSVs into a SQL database, and answer questions about the data. In other words, you will perform:
- Data Engineering
- Data Modeling
- Data Analysis
Inspect the CSVs and sketch out an ERD of the tables; Tables relations docs
-
Use the information you have to create a table schema for each of the six CSV files. Remember to specify data types, primary keys, foreign keys, and other constraints.
- For the primary keys check to see if the column is unique, otherwise create a composite key. Which takes to primary keys in order to uniquely identify a row.
- Be sure to create tables in the correct order to handle foreign keys.
-
Import each CSV file into the corresponding SQL table.
- Here you can find the SQL documentation to Table Schemas
- After Schemas here you can find SQL Query Docs
As you examine the data, you are overcome with a creeping suspicion that the dataset is fake. You surmise that your boss handed you spurious data in order to test the data engineering skills of a new employee. To confirm your hunch, you decide to take the following steps to generate a visualization of the data, with which you will confront your boss:
-
Import the SQL database into Pandas. Here's the SQLAlchemy
-
Create a histogram to visualize the most common salary ranges for employees.
- Create a bar chart of average salary by title.
Evidence in hand, you march into your boss's office and present the visualization. With a sly grin, your boss thanks you for your work. On your way out of the office, you hear the words, "Search your ID number." You look down at your badge to see that your employee ID number is 499942.