diff --git a/Data Science Foundations II/This is Jeopardy/.ipynb_checkpoints/This is Jeopardy-checkpoint.ipynb b/Data Science Foundations II/This is Jeopardy/.ipynb_checkpoints/This is Jeopardy-checkpoint.ipynb new file mode 100644 index 0000000..b5a4cec --- /dev/null +++ b/Data Science Foundations II/This is Jeopardy/.ipynb_checkpoints/This is Jeopardy-checkpoint.ipynb @@ -0,0 +1,222 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# This is Jeopardy!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Overview" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This project is slightly different than others you have encountered thus far. Instead of a step-by-step tutorial, this project contains a series of open-ended requirements which describe the project you'll be building. There are many possible ways to correctly fulfill all of these requirements, and you should expect to use the internet, Codecademy, and/or other resources when you encounter a problem that you cannot easily solve." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Project Goals" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You will work to write several functions that investigate a dataset of _Jeopardy!_ questions and answers. Filter the dataset for topics that you're interested in, compute the average difficulty of those questions, and train to become the next Jeopardy champion!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In order to complete this project, you should have completed the Pandas lessons in the Analyze Data with Python Skill Path. You can also find those lessons in the Data Analysis with Pandas course or the Data Scientist Career Path.\n", + "\n", + "Finally, the Practical Data Cleaning course may also be helpful." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Project Requirements" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. We've provided a csv file containing data about the game show _Jeopardy!_ in a file named `jeopardy.csv`. Load the data into a DataFrame and investigate its contents. Try to print out specific columns.\n", + "\n", + " Note that in order to make this project as \"real-world\" as possible, we haven't modified the data at all - we're giving it to you exactly how we found it. As a result, this data isn't as \"clean\" as the datasets you normally find on Codecademy. More specifically, there's something odd about the column names. After you figure out the problem with the column names, you may want to rename them to make your life easier for the rest of the project.\n", + " \n", + " In order to display the full contents of a column, we've added this line of code for you:\n", + " \n", + " ```py\n", + " pd.set_option('display.max_colwidth', None)\n", + " ```" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "pd.set_option('display.max_colwidth', None)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. Write a function that filters the dataset for questions that contains all of the words in a list of words. For example, when the list `[\"King\", \"England\"]` was passed to our function, the function returned a DataFrame of 49 rows. Every row had the strings `\"King\"` and `\"England\"` somewhere in its `\" Question\"`.\n", + "\n", + " Test your function by printing out the column containing the question of each row of the dataset." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "3. Test your original function with a few different sets of words to try to find some ways your function breaks. Edit your function so it is more robust.\n", + "\n", + " For example, think about capitalization. We probably want to find questions that contain the word `\"King\"` or `\"king\"`.\n", + " \n", + " You may also want to check to make sure you don't find rows that contain substrings of your given words. For example, our function found a question that didn't contain the word `\"king\"`, however it did contain the word `\"viking\"` — it found the `\"king\"` inside `\"viking\"`. Note that this also comes with some drawbacks — you would no longer find questions that contained words like `\"England's\"`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "4. We may want to eventually compute aggregate statistics, like `.mean()` on the `\" Value\"` column. But right now, the values in that column are strings. Convert the`\" Value\"` column to floats. If you'd like to, you can create a new column with float values.\n", + "\n", + " While most of the values in the `\" Value\"` column represent a dollar amount as a string, note that some do not — these values will need to be handled differently!\n", + "\n", + " Now that you can filter the dataset of question, use your new column that contains the float values of each question to find the \"difficulty\" of certain topics. For example, what is the average value of questions that contain the word `\"King\"`?\n", + " \n", + " Make sure to use the dataset that contains the float values as the dataset you use in your filtering function." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "5. Write a function that returns the count of unique answers to all of the questions in a dataset. For example, after filtering the entire dataset to only questions containing the word `\"King\"`, we could then find all of the unique answers to those questions. The answer \"Henry VIII\" appeared 55 times and was the most common answer." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "6. Explore from here! This is an incredibly rich dataset, and there are so many interesting things to discover. There are a few columns that we haven't even started looking at yet. Here are some ideas on ways to continue working with this data:\n", + "\n", + " * Investigate the ways in which questions change over time by filtering by the date. How many questions from the 90s use the word `\"Computer\"` compared to questions from the 2000s?\n", + " * Is there a connection between the round and the category? Are you more likely to find certain categories, like `\"Literature\"` in Single Jeopardy or Double Jeopardy?\n", + " * Build a system to quiz yourself. Grab random questions, and use the input function to get a response from the user. Check to see if that response was right or wrong." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "7. Compare your program to our sample solution code - remember, that your program might look different from ours (and probably will) and that's okay!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "8. Great work! Visit our forums to compare your project to our sample solution code. You can also learn how to host your own solution on GitHub so you can share it with other learners! Your solution might look different from ours, and that's okay! There are multiple ways to solve these projects, and you'll learn more by seeing others' code." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.11" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/Data Science Foundations II/This is Jeopardy/This is Jeopardy.ipynb b/Data Science Foundations II/This is Jeopardy/This is Jeopardy.ipynb new file mode 100644 index 0000000..8a18cfc --- /dev/null +++ b/Data Science Foundations II/This is Jeopardy/This is Jeopardy.ipynb @@ -0,0 +1,376 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# This is Jeopardy!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Overview" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This project is slightly different than others you have encountered thus far. Instead of a step-by-step tutorial, this project contains a series of open-ended requirements which describe the project you'll be building. There are many possible ways to correctly fulfill all of these requirements, and you should expect to use the internet, Codecademy, and/or other resources when you encounter a problem that you cannot easily solve." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Project Goals" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You will work to write several functions that investigate a dataset of _Jeopardy!_ questions and answers. Filter the dataset for topics that you're interested in, compute the average difficulty of those questions, and train to become the next Jeopardy champion!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In order to complete this project, you should have completed the Pandas lessons in the Analyze Data with Python Skill Path. You can also find those lessons in the Data Analysis with Pandas course or the Data Scientist Career Path.\n", + "\n", + "Finally, the Practical Data Cleaning course may also be helpful." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Project Requirements" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. We've provided a csv file containing data about the game show _Jeopardy!_ in a file named `jeopardy.csv`. Load the data into a DataFrame and investigate its contents. Try to print out specific columns.\n", + "\n", + " Note that in order to make this project as \"real-world\" as possible, we haven't modified the data at all - we're giving it to you exactly how we found it. As a result, this data isn't as \"clean\" as the datasets you normally find on Codecademy. More specifically, there's something odd about the column names. After you figure out the problem with the column names, you may want to rename them to make your life easier for the rest of the project.\n", + " \n", + " In order to display the full contents of a column, we've added this line of code for you:\n", + " \n", + " ```py\n", + " pd.set_option('display.max_colwidth', None)\n", + " ```" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Index(['Show Number', ' Air Date', ' Round', ' Category', ' Value',\n", + " ' Question', ' Answer'],\n", + " dtype='object')\n", + "Index(['Show Number', 'Air Date', 'Round', 'Category', 'Value', 'Question',\n", + " 'Answer'],\n", + " dtype='object')\n", + "0 For the last 8 years of his life, Galileo was under house arrest for espousing this man's theory\n", + "1 No. 2: 1912 Olympian; football star at Carlisle Indian School; 6 MLB seasons with the Reds, Giants & Braves\n", + "2 The city of Yuma in this state has a record average of 4,055 hours of sunshine each year\n", + "3 In 1963, live on \"The Art Linkletter Show\", this company served its billionth burger\n", + "4 Signer of the Dec. of Indep., framer of the Constitution of Mass., second President of the United States\n", + " ... \n", + "216925 This Puccini opera turns on the solution to 3 riddles posed by the heroine\n", + "216926 In North America this term is properly applied to only 4 species that are crested, including the tufted\n", + "216927 In Penny Lane, where this \"Hellraiser\" grew up, the barber shaves another customer--then flays him alive!\n", + "216928 From Ft. Sill, Okla. he made the plea, Arizona is my land, my home, my father's land, to which I now ask to... return\"\n", + "216929 A silent movie title includes the last name of this 18th c. statesman & favorite of Catherine the Great\n", + "Name: Question, Length: 216930, dtype: object\n" + ] + } + ], + "source": [ + "import pandas as pd\n", + "pd.set_option('display.max_colwidth', None)\n", + "\n", + "# Loading the data and investigating it\n", + "jeopardy_data = pd.read_csv(\"jeopardy.csv\")\n", + "print(jeopardy_data.columns)\n", + "\n", + "# Renaming misformatted columns\n", + "jeopardy_data = jeopardy_data.rename(columns = {\" Air Date\": \"Air Date\", \" Round\" : \"Round\", \" Category\": \"Category\", \" Value\": \"Value\", \" Question\":\"Question\", \" Answer\": \"Answer\"})\n", + "print(jeopardy_data.columns)\n", + "print(jeopardy_data[\"Question\"])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. Write a function that filters the dataset for questions that contains all of the words in a list of words. For example, when the list `[\"King\", \"England\"]` was passed to our function, the function returned a DataFrame of 49 rows. Every row had the strings `\"King\"` and `\"England\"` somewhere in its `\" Question\"`.\n", + "\n", + " Test your function by printing out the column containing the question of each row of the dataset." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4953 Both England's King George V & FDR put their stamp of approval on this \"King of Hobbies\"\n", + "6337 In retaliation for Viking raids, this \"Unready\" king of England attacks Norse areas of the Isle of Man\n", + "9191 This king of England beat the odds to trounce the French in the 1415 Battle of Agincourt\n", + "11710 This Scotsman, the first Stuart king of England, was called \"The Wisest Fool in Christendom\"\n", + "13454 It's the number that followed the last king of England named William\n", + " ... \n", + "208295 In 1066 this great-great grandson of Rollo made what some call the last Viking invasion of England\n", + "208742 Dutch-born king who ruled England jointly with Mary II & is a tasty New Zealand fish\n", + "213870 In 1781 William Herschel discovered Uranus & initially named it after this king of England\n", + "216021 His nickname was \"Bertie\", but he used this name & number when he became king of England in 1901\n", + "216789 This kingdom of England grew from 2 settlements, one founded around 495 by Cerdic & his son Cynric\n", + "Name: Question, Length: 152, dtype: object\n" + ] + } + ], + "source": [ + "def filter_data(data, words):\n", + " # Convert all words to lowercase for case-insensitive matching\n", + " words = [word.lower() for word in words]\n", + " # Create a filter function that checks if all words are in the question\n", + " filter_func = lambda x: all(word in x.lower() for word in words)\n", + " # Apply the filter to the Question column\n", + " return data[data[\"Question\"].apply(filter_func)]\n", + "\n", + "# Test the function\n", + "filtered = filter_data(jeopardy_data, [\"King\", \"England\"])\n", + "print(filtered[\"Question\"])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "3. Test your original function with a few different sets of words to try to find some ways your function breaks. Edit your function so it is more robust.\n", + "\n", + " For example, think about capitalization. We probably want to find questions that contain the word `\"King\"` or `\"king\"`.\n", + " \n", + " You may also want to check to make sure you don't find rows that contain substrings of your given words. For example, our function found a question that didn't contain the word `\"king\"`, however it did contain the word `\"viking\"` — it found the `\"king\"` inside `\"viking\"`. Note that this also comes with some drawbacks — you would no longer find questions that contained words like `\"England's\"`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "4. We may want to eventually compute aggregate statistics, like `.mean()` on the `\" Value\"` column. But right now, the values in that column are strings. Convert the`\" Value\"` column to floats. If you'd like to, you can create a new column with float values.\n", + "\n", + " While most of the values in the `\" Value\"` column represent a dollar amount as a string, note that some do not — these values will need to be handled differently!\n", + "\n", + " Now that you can filter the dataset of question, use your new column that contains the float values of each question to find the \"difficulty\" of certain topics. For example, what is the average value of questions that contain the word `\"King\"`?\n", + " \n", + " Make sure to use the dataset that contains the float values as the dataset you use in your filtering function." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Average value of 'King' questions: $771.88\n" + ] + } + ], + "source": [ + "def convert_value_to_float(value):\n", + " if value == \"no value\":\n", + " return 0.0\n", + " return float(value.strip('$').replace(',', ''))\n", + "\n", + "jeopardy_data[\"Float Value\"] = jeopardy_data[\"Value\"].apply(convert_value_to_float)\n", + "\n", + "# Calculate average value of questions containing \"King\"\n", + "king_questions = filter_data(jeopardy_data, [\"King\"])\n", + "print(f\"Average value of 'King' questions: ${king_questions['Float Value'].mean():.2f}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "5. Write a function that returns the count of unique answers to all of the questions in a dataset. For example, after filtering the entire dataset to only questions containing the word `\"King\"`, we could then find all of the unique answers to those questions. The answer \"Henry VIII\" appeared 55 times and was the most common answer." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Top 5 answers for 'King' questions:\n", + "Answer\n", + "Henry VIII 55\n", + "Solomon 35\n", + "Richard III 33\n", + "Louis XIV 31\n", + "David 30\n", + "Name: count, dtype: int64\n" + ] + } + ], + "source": [ + "def get_answer_counts(data):\n", + " return data[\"Answer\"].value_counts()\n", + "\n", + "# Test the function with \"King\" questions\n", + "king_answer_counts = get_answer_counts(king_questions)\n", + "print(\"Top 5 answers for 'King' questions:\")\n", + "print(king_answer_counts.head())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "6. Explore from here! This is an incredibly rich dataset, and there are so many interesting things to discover. There are a few columns that we haven't even started looking at yet. Here are some ideas on ways to continue working with this data:\n", + "\n", + " * Investigate the ways in which questions change over time by filtering by the date. How many questions from the 90s use the word `\"Computer\"` compared to questions from the 2000s?\n", + " * Is there a connection between the round and the category? Are you more likely to find certain categories, like `\"Literature\"` in Single Jeopardy or Double Jeopardy?\n", + " * Build a system to quiz yourself. Grab random questions, and use the input function to get a response from the user. Check to see if that response was right or wrong." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "'Computer' questions in the 90s: 98\n", + "'Computer' questions in the 2000s: 327\n", + "\n", + "Top 5 categories in Single Jeopardy:\n", + "Category\n", + "POTPOURRI 255\n", + "STUPID ANSWERS 255\n", + "SPORTS 253\n", + "ANIMALS 233\n", + "AMERICAN HISTORY 227\n", + "Name: count, dtype: int64\n", + "\n", + "Top 5 categories in Double Jeopardy:\n", + "Category\n", + "BEFORE & AFTER 450\n", + "LITERATURE 381\n", + "SCIENCE 296\n", + "WORLD GEOGRAPHY 254\n", + "OPERA 250\n", + "Name: count, dtype: int64\n" + ] + } + ], + "source": [ + "# Compare \"Computer\" questions in 90s vs 2000s\n", + "jeopardy_data['Air Date'] = pd.to_datetime(jeopardy_data['Air Date'])\n", + "nineties = jeopardy_data[(jeopardy_data['Air Date'].dt.year >= 1990) & (jeopardy_data['Air Date'].dt.year < 2000)]\n", + "two_thousands = jeopardy_data[jeopardy_data['Air Date'].dt.year >= 2000]\n", + "\n", + "nineties_computer = filter_data(nineties, [\"Computer\"])\n", + "two_thousands_computer = filter_data(two_thousands, [\"Computer\"])\n", + "\n", + "print(f\"'Computer' questions in the 90s: {len(nineties_computer)}\")\n", + "print(f\"'Computer' questions in the 2000s: {len(two_thousands_computer)}\")\n", + "\n", + "# Check category distribution in different rounds\n", + "single_jeopardy = jeopardy_data[jeopardy_data['Round'] == 'Jeopardy!']\n", + "double_jeopardy = jeopardy_data[jeopardy_data['Round'] == 'Double Jeopardy!']\n", + "\n", + "print(\"\\nTop 5 categories in Single Jeopardy:\")\n", + "print(single_jeopardy['Category'].value_counts().head())\n", + "print(\"\\nTop 5 categories in Double Jeopardy:\")\n", + "print(double_jeopardy['Category'].value_counts().head())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "7. Compare your program to our sample solution code - remember, that your program might look different from ours (and probably will) and that's okay!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "8. Great work! Visit our forums to compare your project to our sample solution code. You can also learn how to host your own solution on GitHub so you can share it with other learners! Your solution might look different from ours, and that's okay! There are multiple ways to solve these projects, and you'll learn more by seeing others' code." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}