workflow-style.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "95f0a171",
   "metadata": {},
   "source": [
    "(workflow-style)=\n",
    "# Workflow: Style\n",
    "\n",
    "Good coding style is like correct punctuation: you can manage without it, butitsuremakesthingseasiertoread. Even as a very new programmer it's a good idea to work on your code style. Use a consistent style makes it easier for others (including future-you!) to read your work, and is particularly important if you need to get help from someone else.\n",
    "\n",
    "This chapter will introduce you to some important style points drawn from [Clean Code in Python](https://testdriven.io/blog/clean-code-python/), [Tips for Better Coding](https://aeturrell.github.io/coding-for-economists/code-best-practice.html) from *Coding for Economists*, the UK Government Statistical Service’s [Quality Assurance of Code for Analysis and Research](https://best-practice-and-impact.github.io/qa-of-code-guidance/intro.html) guidance, and the bible of Python style guides, [PEP 8 — Style Guide for Python Code](https://peps.python.org/pep-0008/).\n",
    "\n",
    "Styling your code will feel a bit tedious to start with, but if you practice it, it will soon become second nature. Additionally, there are some great tools to quickly restyle existing code, like the [Black](https://black.readthedocs.io/) Python package (\"you can have any colour you like, as long as it's black\").\n",
    "\n",
    "Once you've installed Black by running `pip install black`, you can use it on the command line (aka the terminal) within Visual Studio Code. Open up a terminal by clicking 'Terminal -> New Terminal' and then run `black *.py` to apply a standard code style to all Python scripts in the current directory."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8111c37",
   "metadata": {},
   "source": [
    "## Names\n",
    "\n",
    "First, names matter. Use meaningful names for variables, functions, or whatever it is you're naming. Avoid abbreviations that you understand *now* but which will be unclear to others, or future you. For example, use `real_wage_hourly` over `re_wg_ph`. I know it's tempting to use `temp` but you'll feel silly later when you can't for the life of you remember what `temp` does or is. A good trick when naming booleans (variables that are either true or false) is to use `is` followed by what the boolean variable refers to, for example `is_married`.\n",
    "\n",
    "As well as this general tip, Python has conventions on naming different kinds of variables. The naming convention for almost all objects is lower case separated by underscores, e.g. `a_variable=10` or ‘this_is_a_script.py’. This style of naming is also known as snake case. There are different naming conventions though—[Allison Horst](https://twitter.com/allison_horst) made this fantastic cartoon of the different conventions that are in use.\n",
    "\n",
    "![Different naming conventions. Artwork by @allison_horst.](https://github.com/aeturrell/coding-for-economists/raw/main/img/in_that_case.jpg) Different naming conventions. Artwork by @allison_horst.\n",
    "\n",
    "There are three exceptions to the snake case convention: classes, which should be in camel case, eg `ThisIsAClass`; constants, which are in capital snake case, eg `THIS_IS_A_CONSTANT`; and packages, which are typically without spaces or underscores and are lowercase `thisisapackage`.\n",
    "\n",
    "For some quick shortcuts to re-naming columns in **pandas** data frames or other string variables, try the unicode-friendly [**slugify**](https://github.com/un33k/python-slugify) library or the `clean_headers()` function from the [**dataprep**](https://docs.dataprep.ai/index.html) library.\n",
    "\n",
    "The better named your variables, the clearer your code will be--and the fewer comments you will need to write!\n",
    "\n",
    "In summary:\n",
    "- use descriptive variable names that reveal your intention, eg `days_since_treatment`\n",
    "- avoid using ambiguous abbreviations in names, eg use `real_wage_hourly` over `rw_ph`\n",
    "- always use the same vocabulary, eg don't switch from `worker_type` to `employee_type`\n",
    "- avoid 'magic numbers', eg numbers in your code that set a key parameter. Set these as named constants instead. Here's an example:\n",
    "  ```python\n",
    "    import random\n",
    "\n",
    "    # This is bad\n",
    "    def roll():\n",
    "        return random.randint(0, 36)  # magic number!\n",
    "\n",
    "    # This is good\n",
    "    MAX_INT_VALUE = 36\n",
    "\n",
    "    def roll():\n",
    "        return random.randint(0, MAX_INT_VALUE)\n",
    "  ```\n",
    "- use verbs for function names, eg `get_regression()`\n",
    "- use consistent verbs for function names, don't use `get_score()` and `grab_results()` (instead use `get` for both)\n",
    "- variable names should be snake_case and all lowercase, eg `first_name`\n",
    " - class names should be CamelCase, eg `MyClass`\n",
    " - function names should be snake_case and all lowercase, eg `quick_sort()`\n",
    " - constants should be snake_case and all uppercase, eg `PI = 3.14159`\n",
    " - modules should have short, snake_case names and all lowercase, eg `pandas`\n",
    " - single quotes and double quotes are equivalent so pick one and be consistent—most automatic formatters prefer `\"`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b552c344",
   "metadata": {},
   "source": [
    "## Whitespace\n",
    "\n",
    "Surrounding bits of code with whitespace can significantly enhance readability. One such convention is that functions should have two blank lines following their last line. Another is that assignments are separated by spaces\n",
    "\n",
    "```python\n",
    "this_is_a_var = 5\n",
    "```\n",
    "\n",
    "Another convention is that a space appears after a `,`, for example in the definition of a list we would have\n",
    "\n",
    "```python\n",
    "list_var = [1, 2, 3, 4]\n",
    "```\n",
    "\n",
    "rather than\n",
    "\n",
    "```python\n",
    "list_var = [1,2,3,4]\n",
    "# or\n",
    "list_var = [1 , 2 , 3 , 4]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54048af1",
   "metadata": {},
   "source": [
    "## Code Comments\n",
    "\n",
    "As mentioned previously, Python will ignore any text after `#`. This allows to you to write **comments**, text that is ignored by Python but can be read by other humans. Comments can be helpful for briefly describing what the subsequent code does: use them to provide extra contextual information that *isn't* conveyed by function and variable names.\n",
    "\n",
    "Actually, well-written code needs *fewer* comments because it's more evidence what's going on. And it's tempting not to update comments even when code changes. So do comment, but see if you can make the code tell its own story first.\n",
    "\n",
    "Also, avoid \"noise\" comments that tell you what you already know from just looking at the code.\n",
    "\n",
    "![Picture of a cat wearing a label that says cat](https://i.postimg.cc/t4SV5NQg/catto.png)\n",
    "\n",
    "Finally, functions come with their own special type of comments called a doc string. Here's an example that tells us all about the functions inputs and outputs, including the type of input and output (here a data frame, also known as `pd.DataFrame`).\n",
    "\n",
    "```python\n",
    "def round_dataframe(df: pd.DataFrame) -> pd.DataFrame:\n",
    "    \"\"\"Rounds numeric columns in dataframe to 2 s.f.\n",
    "    Args:\n",
    "        df (pd.DataFrame): Input dataframe\n",
    "    Returns:\n",
    "        pd.DataFrame: Dataframe with numbers rounded to 2 s.f.\n",
    "    \"\"\"\n",
    "    for col in df.select_dtypes(\"number\"):\n",
    "        df[col] = df[col].apply(lambda x: float(f'{float(f\"{x:.2g}\"):g}'))\n",
    "    return df\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "861f3102",
   "metadata": {},
   "source": [
    "## Line width and line continuation\n",
    "\n",
    "For quite arbitrary historical reasons, PEP8 also suggests 79 characters for each line of code. Some find this too restrictive, especially with the advent of wider monitors. But it is good to split up very long lines. Anything that is contained in parenthesis can be split into multiple lines like so:\n",
    "\n",
    "```python\n",
    "def function(input_one, input_two,\n",
    "             input_three, input_four):\n",
    "    result = (input_one,\n",
    "              + input_two,\n",
    "              + input_three,\n",
    "              + input_four)\n",
    "    return result\n",
    "```\n",
    "\n",
    "When using *method chaining* (something you can see in action in [](data-transform)) it's necessary to put the chain inside parentheses and it's good practice to use a new line for every method. The code snippet below gives an example of what good looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0f5bb37",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.DataFrame(\n",
    "    data={\n",
    "        \"col0\": [0, 0, 0, 0],\n",
    "        \"col1\": [0, 0, 0, 0],\n",
    "        \"col2\": [0, 0, 0, 0],\n",
    "        \"col3\": [\"a\", \"b\", \"b\", \"a\"],\n",
    "        \"col4\": [\"alpha\", \"gamma\", \"gamma\", \"gamma\"],\n",
    "    },\n",
    "    index=[\"row\" + str(i) for i in range(4)],\n",
    ")\n",
    "\n",
    "\n",
    "# Chaining inside parentheses works\n",
    "\n",
    "results = df.groupby([\"col3\", \"col4\"]).agg({\"col1\": \"count\", \"col2\": \"mean\"})\n",
    "\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d6f3bf8",
   "metadata": {},
   "source": [
    "And this is what *not* to do:\n",
    "\n",
    "```python\n",
    "results = df\n",
    "    .groupby([\"col3\", \"col4\"]).agg({\"col1\": \"count\", \"col2\": \"mean\"})\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d016d530",
   "metadata": {},
   "source": [
    "## Principles of Clean Code\n",
    "\n",
    "While automation can help apply style, it can't help you write *clean code*. Clean code is a set of rules and principles that helps to keep your code readable, maintainable, and extendable. Writing code is easy; writing clean code is hard! However, if you follow these principles, you won't go far wong.\n",
    "\n",
    "### Do not repeat yourself (DRY)\n",
    "\n",
    "The DRY principle is 'Every piece of knowledge or logic must have a single, unambiguous representation within a system.' Divide your code into re-usable pieces that you can call when and where you want. Don't write lengthy methods, but divide logic up into clearly differentiated chunks.\n",
    "\n",
    "This saves having to repeat code, having no idea whether it's this or that version of the same function doing the work, and will help your debugging efforts no end.\n",
    "\n",
    "Some practical ways to apply DRY in practice are to use functions, to put functions or code that needs to be executed multiple times by multiple different scripts into another script (eg called `utilities.py`) and then import it, and to think carefully if another way of writing your code would be more concise (yet still readable).\n",
    "\n",
    "```{admonition} Tip\n",
    ":class: tip\n",
    "If you're using Visual Studio Code, you can [automatically send code into a function](https://code.visualstudio.com/docs/editor/refactoring) by right-clicking on code and using the 'Extract to method' option.\n",
    "```\n",
    "\n",
    "### KISS (Keep It Simple, Stupid)\n",
    "\n",
    "Most systems work best if they are kept simple, rather than made complicated. This is a rule that says you should avoid unnecessary complexity. If your code is complex, it will only make it harder for you to understand what you did when you come back to it later.\n",
    "\n",
    "### SoC (Separation of Concerns) / Make it Modular\n",
    "\n",
    "Do not have a single file that does everything. If you split your code into separate, independent modules it will be easier to read, debug, test, and use. You can check the basics of coding chapter to see how to create and import functions from other scripts. But even within a script, you can still make your code modular by defining functions that have clear inputs and outputs.\n",
    "\n",
    "A good rule of thumb is that if a code that achieves one end goes longer than about 30 lines, it should probably go into a function. Scripts longer than about 500 lines are ripe for splitting up too.\n",
    "\n",
    "Relatedly, do not have a single function that tries to do everything. Functions should have limits too; they should do approximately one thing. If you're naming a function and you have to use 'and' in the name then it's probably worth splitting it into two functions.\n",
    "\n",
    "Functions should have no 'side effects' either; that is, they should only take in value(s), and output value(s) via a return statement. They shouldn't modify global variables or make other changes.\n",
    "\n",
    "Another good rule of thumb is that each function shouldn't have lots of separate arguments.\n",
    "\n",
    "A final tip for modularity and the creation of functions is that you shouldn't use 'flags' in functions (aka boolean conditions). Here's an example:\n",
    "\n",
    "```python\n",
    "# This is bad\n",
    "def transform(text, uppercase):\n",
    "    if uppercase:\n",
    "        return text.upper()\n",
    "    else:\n",
    "        return text.lower()\n",
    "\n",
    "# This is good\n",
    "def uppercase(text):\n",
    "    return text.upper()\n",
    "\n",
    "def lowercase(text):\n",
    "    return text.lower()\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "9d7534ecd9fbc7d385378f8400cf4d6cb9c6175408a574f1c99c5269f08771cc"
  },
  "jupytext": {
   "cell_metadata_filter": "-all",
   "encoding": "# -*- coding: utf-8 -*-",
   "formats": "md:myst",
   "main_language": "python"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "toc-showtags": true
 },
 "nbformat": 4,
 "nbformat_minor": 5
}