Introduction

Overview

Teaching: 30 min
Exercises: 15 min

Questions

• What are The Turing Way and Jupyter Book projects?

Objectives

• Explain what The Turing Way project is and how it allows people to exchange computational skills and best practices for reproducible research

• Explain what the Jupyter Book project is and how it allows collaborative development of an online book

• Demonstrate of The Turing Way GitHub repository and show how its Jupyter Book looks like online

Figures 1: One image description of The Turing Way

The Turing Way

📹 Overview video: The Turing Way

• A book:
  • https://the-turing-way.netlify.com/
• An open source project:
  • https://github.com/alan-turing-institute/the-turing-way
• A community:
  • Divrse researchers come together to collaborate and develop The Turing Way materials online via GitHub (see Contributors Table)
• A culture of collaboration:

  The Turing Way is an Open Source collaborative project that involves and supports its members of diverse skills and backgrounds to ensure that data science is accessible and useful for everyone. Its moonshot goal is to make reproducible research ‘too easy not to do’.

Reproducibility

At The Turing Way we define reproducible research as work that can be independently recreated from the same data and the same code that the original team used. Reproducible is distinct from replicable, robust and generalisable as described in the figure below.

Figures 2: A result is reproducible when the same analysis steps performed on the same dataset consistently produces the same answer.

Community members sharing best practices for reproducible research

The Turing Way is a community-led book project that involves diverse perspectives of researchers, funders, educators, learners and various stakeholders from around the world.

Our goal is to provide a platform where these members can come together to share skills and resources that data scientists in academia, industry, government and other sectors need at the start of their projects to ensure that their projects are easy to reproduce and reuse at the end.

The book started as a guide for reproducibility, covering technical methods such as version control, testing, and continuous integration. However, technical skills constitute only one aspect of reproducible research. In February 2020, The Turing Way expanded to a series of guides covering topics on reproducible research, project design, communication, collaboration, and ethical research.

To date, there are 100 subchapters, collaboratively written by over 175 contributors and read by hundreds of researchers worldwide.

References:

• GitHub
• Contribution guideline

JupyterBook

📹 VIDEO

Jupyter Book is an open source project for building beautiful, publication-quality books and documents from computational material.

Jupyter Book provides the following features for creating online books:

Publication-quality content in markdown files

• Files can be written in Markdown syntax
• Jupyter Book also proves an extended flavor of markdown with publishing features that allows citations and cross-references, math and equations, and figures that can be resized.

Interactive and executable codes in Jupyter Notebooks

• Chapters can be written in Jupyter Notebooks, which can include code that can be executed while building the book, outputs for which can be displayed in-line in your book.
• The Jupyter visualisation can allow readers to interact with the file content.

Navigable and collaborative workflow

• Jupyter Books are easy to navigate (see demo in the section).
• The GitHub configuration allows readers to collaborate easily by suggeting edits and reporting bugs.

Figure 3: Jupyter Book offers a detailed guide for creating different types of book contents

The guide for using Jupyter Book is written in online Jupyter Book: https://jupyterbook.org/intro.html. It provides guidelines with example projects that one can play with to create a different types of book content and test its features.

References

• GitHub
• Contribution guideline

Project Demos

Setting to explore the Jupyter Book of The Turing Way

Jupyter Book is used for creating the online book of The Turing Way. Please use this link to follow the book demo: https://the-turing-way.netlify.app/welcome In this demo we show you the following aspects of online book of The Turing Way:

5 guides in The Turing Way

• Reproducible Research
• Project Design
• Communication
• Collaboration
• Ethical Research

The Turing Way Community Handbook

The Turing Way Community Handbook aims to provide information about the project, ways of working, and other aspects that can make community participation equitable for our members.

Navigable and collaborative features of Jupyter Book

Figure 4: Jupyter Book feature makes _The Turing Way_ navigable and collaborative.

Users of Jupyter Book can not only read the chapters online, but can easily navigate its file and collaborate in the project.

• As shown in the Figure 4, Jupyter Book creates a table of content for the book in the left (Figure 4.1) and table of content for the chapter for the chapter on the right (Figure 4.5), making the book and chapter contents easy to navigate.
• The book has been configured to allow a reader to read the book in the fullscreen mode (Figure 4.2).
• The collaborative workflow has been explained in the Module-7 of this tutorial.

The Turing Way GitHub repository

We welcome all contributions to this project via issues and pull requests on outr online GitHub repository. We have a contributors guideline that allows our contributors to make sure that their contributions can be easily integrated into the projects.

Important components for building Jupyter Book of The Turing Way

All the content of the book is stored in the book/website folder. Please take a moment to check out these files - you don’t need to read them all, but take note of where they are and what their format is:

1. Chapters contents written in Markdown
2. The _toc.yml file to structure the book by defining the table of content
3. A directory to store all the figures (figures)
4. A bibtex file to store all the references (references.bib)
5. The _config.yml file for configuring the book

In the later modules, we will learn about creating a Jupyter Book from scratch using these important components of the book. We will also see how one can cite an external resources through references.bib file and cross reference chapters within the JupyterBook.

Key Points

• The Turing Way is an open source and collaborative project for reproducible research that provides a common platform for different stakeholders of research to come together and exchange skills around reproducible, ethical, collaborative and inclusive research

• Jupyter Book is an open source projects that can be used for creating interactive computational or non-computational resources to share and develop them online

• The Turing Way uses Jupyter Book for creating and sharing its materials by involving its community members online

• Contents of The Turing Way are collaboratively developed by contributors on project’s GitHub repository

• There are a few important components (files) that are needed to build a The Turing Way Jupyter Book: chapters written in Markdown files, figures folder where all the images used in the book are used, a references.bib file where all the referenced materials are stored in bibtex format, a _toc.yml file that allows how to structure our chapters online and a book folder where all these components of the book are stored.

Creating a Jupyter Book with The Turing Way

Overview

Teaching: 30 min
Exercises: 15 min

Questions

• How to set up the repository, add a chapter, create a table of content, and build the book?

Objectives

• Explain what files exist in the repository that we will use for the hands on session in this module (if you haven’t already, please download the data required for thie tutorial described in the module 1)

• Explore the important components for creating Jupyter Book in this tutorial

• Build the first minimal version of the Jupyter Book locally using example files from The Turing Way

Introduction to Jupyter Book

Welcome! In this Jupyter Notebook we will introduce the basic commands to generate your first Jupyter Book.

In the previous module, we briefly looked into the awesome and very detailed documentation of Jupyter-Book, and its GitHub repository.

Jupyter Book has a command-line interface, so in this tutorial we will show you how to build your book using shell commands.

If you prefer using Jupyter Notebbok, you can use iPython’s special syntax that lets you execute shell commands from a Jupyter Notebook. You can run all the commands used in this tutorial by prefixing the shell command with ! in each cell.

TIP

If you are unfamiliar with executing shell commands from Jupyter Notebooks, read this tutorial by Jake VanderPlas.

Creating the content of your book based on The Turing Way

In order to build our Jupyter Book, we first need to create a folder where all the content of your book exists. This is also where Jupyter Book will create the html files in order to host the book online. As demonstrated for The Turing Way, we will store all the content of our book in a folder named book located in the main repository (it doesn’t need to be named this way for Jupyter Book to work). Let’s create it:

mkdir ../book/

The folder should be empty as you have just created it!

NOTE: If this folder already exists, you are probably re-running this notebook without having cleared the book folder. Make sure you remove it using this command:

$ rm -R book/*

Example files for this tutorial

We will now add some contents such as a book chapter and other important components to our book folder.

In this tutorial, we will be using some of the chapters from The Turing Way. We have provided some example files, which are placed under the folder content in the main repository.

Let’s inspect whats inside it:

ls ../content

There are 2 Markdown files: welcome.md and reproducible-research.md. These are welcome page and introduction to the Guide for Reproducible Research in The Turing Way respectively.

The references.bib is a bibtex file, which contains all the references and a figures folder, which contains all the figures used in these files. To read more about citing using sphinx and Jupyter Book head here.

There are also two folders, overview and open, that contain subchapters of the overview and open science chapters respectively (you can see them in the online The Turing Way book).

Let’s start!

For our first book we don’t want all these files. Let’s import to our bookfolder only the necessary files one by one.

We first need an index page for our book, which is welcome.md file in The Turing Way. Let’s copy it into our book folder:

cp ../content/welcome.md ../book/

We will now need a chapter. Let’s use the second Markdown file reproducible-research.md as our first chapter.

cp ../content/reproducible-research.md ../book/reproducible-research.md

These files link to some citations and images placed in the references.bib file and figures folder, which we will also copy to our book folder:

cp -R ../content/figures/ ../book/figures

cp ../content/references.bib ../book/references.bib

Let’s make sure our book folder now contains all the files we want:

ls ../book/

You should see printed a figures, references.bib, welcome.md and reproducible-research.md file.

We are almost there!

But we can not yet build our Jupyter Book. We have to specify the structure of our book - how would your book look like. Let’s do this in the next section.

Creating the structure of your Jupyter Book (_toc.yml)

A book always have a structure, that is, a Table Of Contents (TOC) that defines the hierarchy and order of the content-files.

Similarly, to create a Jupyter Book we need to have a file that specifies the TOC. Jupyter Book defines the TOC using a yaml file that needs to be named _toc.yml. This can be created manually, or automatically by running:

$ jupyter-book toc {path}

as long as the intended structure is also ordered alphanumerically.

But in The Turing Way we are always creating new content and restructuring our book. In these cases, is not practical to have our filenames ordered alphanumerically. We will thus show you how to manually create our _toc.yml using the python library ruamel.yaml.

Create a file name create_toc.py in your current directory and copy-paste the following code:

from ruamel.yaml import YAML

yaml = YAML()

# Define the contents of our _toc.yml
toc_document = """
- file: welcome
- file: reproducible-research
"""

# Save _toc.yml in the book directory
toc_file = open('../book/_toc.yml', 'w')
yaml.dump(yaml.load(toc_document), toc_file)

If using the Jupyter Notebook, you can run this script directly in a code cell.

Key Points

• • Jupyter Book uses markdown files to create chapters of an online book. In this tutorial, these files are placed in a directory called book.

• All the images used in the file should also be stored in book (we saved them under the folder figures in this tutorial).

• References used in the book should be stored in a references.bib file, which should also be stored in book

• A _toc.yml should be generated inside the folder book to structure the jupyter Book (toc is abbreviation for table of content)

• With these files in place, you can create a minimal book using the command $ jupyter-book build {path-to-book}

Introduction

Overview

Teaching: 30 min
Exercises: 15 min

Questions

Objectives

Key Points

Introduction

Overview

Teaching: 30 min
Exercises: 15 min

Questions

Objectives

Key Points

Introduction

Overview

Teaching: 30 min
Exercises: 15 min

Questions

Objectives

Key Points

Introduction

Overview

Teaching: 30 min
Exercises: 15 min

Questions

Objectives

Key Points