{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "27f43f21",
   "metadata": {},
   "source": [
    "# Example: Billionaire Demographics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11cc74b9",
   "metadata": {},
   "source": [
    "This data visualization tutorial tackles a common pair of data visualization objectives: *1)* showing how data between categories relate and *2)* showing how data within each category are structured.\n",
    "\n",
    "To this end, we'll build a visualization for the demographics of the most wealthy members of different industries.\n",
    "Data for this exercise comes from [Joy Shill via Kaggle](https://www.kaggle.com/datasets/joyshil0599/exploring-wealth-forbes-richest-people-dataset).\n",
    "\n",
    "In our visualization, we will look at how the demographics of the upper echelons of different industries are structured and how that structure compares between industries.\n",
    "Scatterploting age versus wealth faceted per industry will provide a good view of individual industries' top-tier compositions.\n",
    "To aid comparison of demographics between industries, we'll provide a summary plot and structure our industry-specific facets zoomed outsets of the summary plot.\n",
    "We'll arrange the main plot and the zoom outsets together as an axes grid.\n",
    "\n",
    "To take on this latter task --- composing a primary axes with a faceted grid of zoomed views  --- we'll make use of  `outset` library, which provides convenient tools to this end.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce4c5068",
   "metadata": {},
   "source": [
    "Let's start coding!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e0677d1",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe3126bc",
   "metadata": {},
   "source": [
    "Begin by importing necessary packages.\n",
    "\n",
    "Notably:\n",
    "- `adjustText` for nudging scatterplot text labels to prevent overlap\n",
    "- `matplotlib` as the graphics engine\n",
    "- `outset` for managing multi-zoom grid and zoom indicators\n",
    "- `pandas` for data management"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "676f9fd7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import typing\n",
    "\n",
    "from adjustText import adjust_text\n",
    "import outset as otst\n",
    "from outset import patched as otst_patched\n",
    "from matplotlib import pyplot as plt\n",
    "import pandas as pd\n",
    "\n",
    "plt.style.use(\"bmh\")  # aesthetics: switch matplotlib style sheet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ad12b89",
   "metadata": {},
   "source": [
    "To install dependencies for this exercise,\n",
    "\n",
    "```bash\n",
    "python3 -m pip install \\\n",
    "    adjustText `# ==0.8` \\\n",
    "    matplotlib `# ==3.8.2`\\\n",
    "    numpy `# ==1.26.2` \\\n",
    "    outset `# ==0.1.4` \\\n",
    "    opytional `# ==0.1.0` \\\n",
    "    pandas `# ==2.1.3`\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4221755d",
   "metadata": {},
   "source": [
    "## Data Preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d48041b",
   "metadata": {},
   "source": [
    "Next, let's fetch our data and do a little work on it.\n",
    "We'll need to clean up the \"Net Worth\" column, which is polluted with dollar signs and textual \"billion\" qualifiers.\n",
    "We'll also shorten the names of the people in our dataset so they're easier to plot and rank everyone within their own industry in order to isolate the upper echelons of each industry."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d86dbbfa",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"https://osf.io/bvrjm/download\")  # download data\n",
    "df[\"Net Worth (Billion USD)\"] = (\n",
    "    # strip out pesky non-numeric characters then convert to float\n",
    "    df[\"Net Worth\"]\n",
    "    .str.replace(r\"[^.0-9]+\", \"\", regex=True)\n",
    "    .astype(float)\n",
    ")\n",
    "df[\"Who\"] = (  # shorten names so they're easier to plot...\n",
    "    df[\"Name\"]\n",
    "    .str.replace(r\"\\s*&.*\", \"\", regex=True)  # ... remove \"& family\"\n",
    "    .replace(r\"(\\b[A-Za-z])\\w+\\s+\", r\"\\1. \", regex=True)  # abbrev F. Lastname\n",
    "    .str.slice(0, 12)  # chop long names to 12 characters\n",
    ")\n",
    "\n",
    "# rank everyone by wealth within their industry\n",
    "df[\"Industry Rank\"] = df.groupby(\"Industry\")[\"Net Worth (Billion USD)\"].rank(\n",
    "    method=\"dense\", ascending=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbbe0f27",
   "metadata": {},
   "source": [
    "For tractability, we will visualize only a subset of industry categories.\n",
    "Here's a few with interesting contrast in demographic structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ac577c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "focal_industries = [\n",
    "    \"Technology\",\n",
    "    \"Fashion & Retail\",\n",
    "    \"Sports\",\n",
    "    \"Finance & Investments\",\n",
    "    \"Automotive\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb6e69cc",
   "metadata": {},
   "source": [
    "## Setup Axes Grid"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb77df05",
   "metadata": {},
   "source": [
    "Now it's time to get started plotting.\n",
    "We can use an the outset library's `OutsetGrid` class to manage content and layout of our faceted axes grid.\n",
    "This tool operates analogously to seaborn's [`FacetGrid`](https://seaborn.pydata.org/generated/seaborn.FacetGrid.html), if you're familiar with that.\n",
    "\n",
    "We pass `OutsetGrid` a data frame containing our data, the names of the columns we want to use for the x and y axes, and the names of the columns we want to use to split our data into facets.\n",
    "Passing the `hue` kwarg will supplement faceting with color-coding by industry.\n",
    "Other kwargs provide styling and layout information, including how we want plots to be shaped and how many columns we want to have."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f616bc3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "grid = otst.OutsetGrid(  # setup axes grid manager\n",
    "    df[(df[\"Industry Rank\"] < 8)].dropna(),  # only top 8 in each industry\n",
    "    x=\"Age\",\n",
    "    y=\"Net Worth (Billion USD)\",\n",
    "    col=\"Industry\",\n",
    "    hue=\"Industry\",\n",
    "    col_order=focal_industries,  # subset to our focal industries\n",
    "    hue_order=focal_industries,\n",
    "    aspect=1.5,  # widen subplots\n",
    "    col_wrap=3,\n",
    ")\n",
    "\n",
    "pass  # sponge up last return value, if any"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8092cbee",
   "metadata": {},
   "source": [
    "## Set Up Plot Content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4e5ea3f",
   "metadata": {},
   "source": [
    "Next, we'll scatterplot our billionaire's ages and wealths.\n",
    "We need to add each billionaire to the main plot and to their industry's outset plot.\n",
    "\n",
    "We'll use a patched version of seaborn's `scatterplot` function bundled with `outset` due to an [open seaborn issue](https://github.com/mwaskom/seaborn/issues/3601).\n",
    "We will use `OutsetGrid.map_dataframe` to plot appropriate data and colors on each axes.\n",
    "The `map_dataframe` method works analogously it its equivalent on seaborn's `FacetGrid.\n",
    "The first argument provides a plotter function, which is called with subsequent arguments forwarded on each axes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6bb1eb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "grid.map_dataframe(  # map scatterplot over all axes\n",
    "    otst_patched.scatterplot,\n",
    "    x=\"Age\",\n",
    "    y=\"Net Worth (Billion USD)\",\n",
    "    legend=False,\n",
    ")\n",
    "\n",
    "display(grid.figure)  # show current progress\n",
    "\n",
    "pass  # sponge up last return value, if any"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7896b67",
   "metadata": {},
   "source": [
    "## Annotate Outset Plots with Names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb38c597",
   "metadata": {},
   "source": [
    "Our next task is to identify plotted billionaires by name by labeling scatter points on outset plots.\n",
    "To get a good result, some care must be taken to avoid overlapping text labels with each other or with the plotted points.\n",
    "The awesome [adjustText](https://github.com/Phlya/adjustText) library can help us here by automatically nudging text labels to avoid overlap.\n",
    "\n",
    "Let's put together a data-oriented helper function to do all this that we can map over the outset axes in our grid."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4df99682",
   "metadata": {},
   "outputs": [],
   "source": [
    "def annotateplot(\n",
    "    data: pd.DataFrame,\n",
    "    *,\n",
    "    x: str,\n",
    "    y: str,\n",
    "    text: str,\n",
    "    ax: typing.Optional[plt.Axes] = None,\n",
    "    adjusttext_kws: typing.Optional[typing.Mapping] = None,\n",
    "    **kwargs: dict,\n",
    ") -> plt.Axes:\n",
    "    \"\"\"Annotate a plot coordinates with text labels, then apply adjustText to\n",
    "    rearrange the labels to avoid overlaps.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    data : pd.DataFrame\n",
    "        The DataFrame containing the data to plot.\n",
    "    x : str\n",
    "        The name of the column in `data` to use for the x-axis values.\n",
    "    y : str\n",
    "        The name of the column in `data` to use for the y-axis values.\n",
    "    text : Optional[str], default None\n",
    "        The name of the column in `data` to use for text values.\n",
    "    ax : Optional[plt.Axes], default None\n",
    "        The matplotlib Axes object to draw the plot onto, if provided.\n",
    "    adjusttext_kws : Mapping, default {}\n",
    "        Additional keyword arguments forward to adjustText.\n",
    "    **kwargs : dict\n",
    "        Additional keyword arguments forward to seaborn's regplot.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    plt.Axes\n",
    "        The matplotlib Axes containing the plot.\n",
    "    \"\"\"\n",
    "    if adjusttext_kws is None:\n",
    "        adjusttext_kws = {}\n",
    "\n",
    "    if ax is None:\n",
    "        ax = plt.gca()\n",
    "\n",
    "    kwargs.pop(\"legend\", None)  # ignore these kwargs\n",
    "    kwargs.pop(\"label\", None)\n",
    "\n",
    "    texts = [  # add text for each row in data\n",
    "        ax.text(row[x], row[y], row[text], **kwargs)\n",
    "        for _idx, row in data.iterrows()\n",
    "    ]\n",
    "    adjust_text(texts, ax=ax, **adjusttext_kws)\n",
    "\n",
    "    return ax"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d98742f",
   "metadata": {},
   "source": [
    "A call to `OutsetGrid.map_dataframe_outset` applies this function to just the outset axes in our grid."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e8106b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "grid.map_dataframe_outset(  # map name annotations over all outset axes\n",
    "    otst_patched.annotateplot,\n",
    "    x=\"Age\",\n",
    "    y=\"Net Worth (Billion USD)\",\n",
    "    text=\"Who\",\n",
    "    fontsize=8,  # make text slightly smaller so it's easier to lay out\n",
    "    rotation=-5,\n",
    "    adjusttext_kws=dict(  # tweak fiddly params for text layout solver\n",
    "        avoid_self=True,\n",
    "        autoalign=True,\n",
    "        expand_points=(1.8, 1.3),\n",
    "        arrowprops=dict(arrowstyle=\"-\", color=\"k\", lw=0.5),\n",
    "        expand_text=(1.8, 2),\n",
    "        force_points=(1.2, 1),\n",
    "        ha=\"center\",\n",
    "        va=\"top\",\n",
    "    ),\n",
    ")\n",
    "\n",
    "display(grid.figure)  # show current progress\n",
    "\n",
    "pass  # sponge up last return value, if any"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adb56953",
   "metadata": {},
   "source": [
    "## Add Zoom Indicators"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe34dc5f",
   "metadata": {},
   "source": [
    "Now it's time to add zoom indicator boxes, a.k.a. `outset` \"marquees,\" to show how the scales of our auxiliary plots relate to the scale of the main plot.\n",
    "Note that we pass the `equalize_aspect` kwarg so that aspect ratios can vary between the main plot and outset plots.\n",
    "That way, zoom areas will expand to take full advantage of available space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41fd80f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# draw \"marquee' zoom indicators showing correspondences between main plot\n",
    "# and outset plots\n",
    "grid.marqueeplot(equalize_aspect=False)  # allow axes aspect ratios to vary\n",
    "\n",
    "display(grid.figure)  # show current progress\n",
    "\n",
    "pass  # sponge up last return value, if any"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72abca13",
   "metadata": {},
   "source": [
    "## Et Voilà!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a806b5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "display(grid.figure)  # show completed figure!\n",
    "\n",
    "pass  # sponge up last return value, if any"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b6bbc64",
   "metadata": {},
   "source": [
    "## Citations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eef970c6",
   "metadata": {},
   "source": [
    "> Ilya Flyamer, Zhuyi Xue, Colin, Andy Li, JasonMendoza2008, Josh L. Espinoza, Nader Morshed, Oscar Gustafsson, Victor Vazquez, Ryan Neff, mski_iksm, Nikita Vaulin, scaine1, & Oliver Lee. (2023). Phlya/adjustText: 0.8.1 (0.8.1). Zenodo. https://doi.org/10.5281/zenodo.10016869\n",
    "\n",
    "> J. D. Hunter, \"Matplotlib: A 2D Graphics Environment\", Computing in Science & Engineering, vol. 9, no. 3, pp. 90-95, 2007. https://doi.org/10.1109/MCSE.2007.55\n",
    "\n",
    "> Matthew Andres Moreno. (2023). mmore500/outset. Zenodo. https://doi.org/10.5281/zenodo.10426106\n",
    "\n",
    "> Data structures for statistical computing in python, McKinney, Proceedings of the 9th Python in Science Conference, Volume 445, 2010. https://doi.org/ 10.25080/Majora-92bf1922-00a\n",
    "\n",
    "> Joy Shil. (2023). Exploring Wealth: Forbes Richest People Dataset [Data set]. Kaggle. https://doi.org/10.34740/KAGGLE/DSV/5272751\n",
    "\n",
    "> Waskom, M. L., (2021). seaborn: statistical data visualization. Journal of Open Source Software, 6(60), 3021, https://doi.org/10.21105/joss.03021."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}