Quarto

Le mot de la fin

today

Hello Quarto

share • collaborate • teach • reimagine
(un Notebook pour les gouverner tous)

Aurélie Mercadié
Christophe Regouby

camembr.quarto.pub/hello-quarto

artwork by allison horst

Une
his-
toire
du
Notebook

Une définition

Une définition

Le Notebook, c’est la rencontre entre

• l’exécution du code
  

• la narration
  
  • l’intention
    
  • l’approche
    
  • les résultats
    

Quelques exemples

Quelques exemples

Quelques exemples

::: :::

Quelques exemples

Les publications

Les packages

Les packages

tout un écosystème…

Hello, Quarto?

Quarto unifie et étend
l’écosystème R Markdown

Quarto unifies & extends the R Markdown ecosystem.

Quarto unifie et étend
l’écosystème R Markdown

unifie les fans de R Markdown

étend l’écosysteme à ceux qui ne connaissent pas R Markdown

It unifies it for people who love R Markdown by reducing little points of friction, so that websites, books, and slides all have a common approach.

And it extends it for people who don’t know RMarkdown by being a friendly way to work reproducibly and publish documents that have text and code in the same place

Quarto se présente comme le nouveau système open-source
de publication d’articles scientifiques and techniques

Quarto is a new open-source scientific and technical publishing system.

Quarto se présente comme le nouveau système open-source
de publication d’articles scientifiques and techniques

dans le but de rendre le process de création
et de collaboration radicalement plus simple

The goal is to make the process of creating and collaborating dramatically better.

–> It handles inputs in multiple languages, and outputs in multiple formats, all going through the same machinery.

–> In our talk today, we’ll introduce you to Quarto with stories and demos.

First Mine will focus on how Quarto unifies the R Markdown ecosystem through stories of sharing and teaching

Then I will focus on how Quarto extends the R Markdown ecosystem through collaborating and reimagining

Share

The R Markdown ecosystem

Julie mentioned unifying the R Markdown ecosystem so let’s dig a bit more into what that means…

Think of all the packages from the R Markdown universe that you’ve come to love and rely on over the years. Each package addresses a different need and offers a different output: for example, you can use blogdown or distill for creating websites and blogs, bookdown for writing a book, rticles for writing journal articles, etc.

Quarto: Next generation R Markdown

Quarto unifies the functionality from these packages,

building on Pandoc on the technical side

and on the human side – over a decade of experience developing, maintaining, and tweaking these packages as well as community feedback R Markdown extension packages.

Such a unification effort presents a fantastic opportunity to identify and address gaps functionality and inefficiencies in user experience, and the process of building Quarto has been no different.

Quarto highlights

Consistent implementation of attractive and handy features across outputs: tabsets, code-folding, syntax highlighting, etc.

More accessible defaults as well as better support for accessibility

Guardrails, particularly helpful for new learners: YAML completion, informative syntax errors, etc.

Support for other languages like Python, Julia, Observable, and more via Jupyter engine for executable code chunks.

Some highlights of these improvements include

–> consistent implementation of attractive and handy features across all outputs, like tabsets, code-folding, and syntax highlighting,

–> more accessible defaults as well as better support for creating accessible documents,

–> guardrails that are particularly helpful for new learners, like YAML completion and informative syntax errors,

–> and perhaps most excitingly for those who are not coming from the R ecosystem, Quarto offers support for other languages like Python, Julia, Observable, and more via the Jupyter engine for executable code chunks. And it’s designed to be expandable to more engines and languages, even those that might not exist today.

[pause] So by now I’m assuming many of you have already started playing with Quarto, or you’re waiting for me to do that. So, let’s dive in and see what all the Quarto fuss is about!

• Open hello.Rmd with source editor -> knit

• Change to visual editor -> knit

• Close file -> change file type to .qmd -> open + render

• Change output: html_document to format: html -> render

• Click on Render on save

• Change to format: pdf -> render -> then change back

• Add chunk option echo: false to the first chunk

• Add execute > echo: false to the yaml

• Add alt text to figure in second code chunk -> save to render , fig.alt=“Scatterplot of bill length vs. flipper length for three species of penguins.”

• Change to format: revealjs -> save to render

• Add H2 to break slides

• Change to format: html

• Add index.qmd with File > New Quarto Document

• Add _quarto.yml for website

• Relaunch project

• Show new Build tab

• Render site, show _site folder

• Add freeze to _quarto.yml

• From terminal: quarto publish

Quarto CLI orchestrates
each step of rendering

Now that you’ve all had a chance to see Quarto in action, you might be wondering, “how are R Markdown and Quarto different”?

The main difference is that with R Markdown, the R package rmarkdown does the heavy lifting of going from the source Rmd file to the the output, using knitr for evaluating the code chunks.

–> With Quarto, on the other hand, the Quarto command line interface, or the Quarto CLI, does the orchestration of processing executable code chunks with either knitr or jupyter and then converting the resulting markdown file to the desired output.

While this is technically impressive, I’ll be honest, it’s not exactly what sparked my interest in Quarto in the first place.

Quarto makes moving between formats straightforward

Document

lesson-1.qmd

title: "Lesson 1"
format: html

Presentation

lesson-1.qmd

title: "Lesson 1"
format: revealjs

Website

_quarto.yml

project:
  type: website

website: 
  navbar: 
    left:
      - lesson-1.qmd

What did spark my interest was how how straightforward it is with Quarto to move between output formats.

As an educator, two things are of utmost importance to me about the tools I use to create my materials: reproducibility and ease of transition between output formats, like documents to slides to websites to books.

Over the last year of using Quarto for pretty much everything, I’ve felt like I’ve finally found the tool that lets me go from one output type to the other with minimal, if any, futzing around with my source code beyond the yaml. For example, here are common things I produce

–> a lesson in document form
–> the same content in presentation form
–> the same content on a page in a website,

and you can see that all that needed to change going between these formats is a few lines in the yaml. Nothing in the content part of my document. No slide breaks to remove, no citation style to change, no headings to re-level. This ease of transition has freed up time to focus my time on content, and that, folks, is the dream!

Quarto facilitates open sharing

Tooling that makes it easy to share what you do
with the world on the web is incredibly useful

It’s useful not just to you, but to the world
as it facilitates open sharing of knowledge
for learning and for reuse

Ultimately, easily creating reproducible artifacts is great.

But for the sharing story to be complete, the tool also needs to support easily sharing those artifacts, like with a simple command like quarto publish, and hopefully soon a button RStudio that does the same thing.

–> I think that tooling that makes it easy to share what you do with the world on the web is incredibly useful.

–> And it’s not just useful to you, the creator, but to the world, as it facilitates open sharing of knowledge for learning and for reuse.

[Pause]

I’ve been talking just about me and my work so far, but I, like many of you, don’t operate in a silo disconnected from others.

[Aurelie continue on next slide]

Collaborate

Many Data scientists
use Python
😬

Many data scientists use python and jupyter notebooks. And my role is not to learn python from the ground-up, but I needed to be able to collaborate with and support python users.

Transition: But I also brought over 6 years of experience and confidence that I’ve developed using RMarkdown. I think of RMarkdown like a superpower

And Quarto let me bring this superpower to python.

Quarto enables collaborating
across coding languages

_quarto.yml

contents:
  - section: in-development/index.qmd
    contents:
      - in-development/earthdata-access-demo.ipynb
      - in-development/nsidc/icesat2-cmr-onprem-vs-cloud.ipynb
      - in-development/lpdaac/lpdaac_ecostress_lste_kerchunk.ipynb
      - in-development/matlab-aws.qmd
      - in-development/earthdata-python-r-handoff.rmd  
  - section: contributing/onboarding.qmd
    contents: 
      - contributing/quarto-website-tutorial.md

Quarto enables collaborating across coding languages. This is a simplified view of the yaml it’s collection of different file types. It’s mostly ipynbs Jupyter notebooks.

And, as we work with more research teams we’ve found that there is indeed a substantial R community using NASA Earthdata Cloud data. And we’re positioned to support R users too

Quarto enables contributing
from our current tools

we’re all able to contribute from whatever tool we’re already comfortable with. In our group some use RStudio, some use JupyterLab some VS Code, some directly edit documents on GitHub.

And this was important because of all the new things we were learning together, we didn’t also need people to shift their software tools.

transtion: Let me show you how to use Quarto with Jupyter with ipynb notebooks

• Orient to JupyterLab. This is a JupyterHub managed by our collaborators at 2i2c
• Open Mine’s example site, quarto preview
• add .ipynb
• Add .ipynb to _quarto.yml
• View website - new page is added to website
• Add fig captions
• margin plots: a style preferred by many now in jupyter notebook

Teach

For learning:

• RStudio comes with everything, no additional install needed
• visual editor – Julie will say more
• code-link
• yaml errors

For teaching:

• revealjs
• Show hamburger menu outline and print to PDF
• chalkboard
• multiplex: Slides / scroll for audience
• execute > echo: true
• Add code line highlighting to plotting chunk

Features that are useful for research:

• quarto journals

Quarto flattens the learning curve

So, in regards to teaching, I’ve talked about a much less steep hill students need to climb as their projects grow. I’ve also said this was a pleasant surprise as it was not something I had thought about when I started teaching with Quarto. But, in hindsight, this was no coincidence. Quarto’s features like consistent citation syntax, more intentional control over when computations should run or rerun, and a single, minimal yaml that controls inputs, layouts, theming, etc. all contribute to flattening this learning curve.

Quarto can grow with learners in a profound way

And beyond learners growing and expanding the scope of their work easily with Quarto, Quarto can grow with them in a profound way as well.

–> The fact that Quarto is language and authoring tool independent offers an opportunity for folks to take Quarto with them on their data science journey, wherever that journey takes them. I mean.. I’m almost tempted to finally learn Python to see it in action in Quarto. But who I know is learning Python, and Julia, and other languages are my students in their Computer Science courses. To imagine that they could learn R in RStudio using Quarto in my course, and then continue to use Quarto in Jupyter in their CS courses… That sounds like such a bright future to me!

[Pause] And a bright future includes things we can’t even imagine yet.

[Mine continue on next slide]

camembr.quarto.pub/hello-quarto

If you would like to start sharing, collaborating, teaching, and reimagining with Quarto by picking up where we left off, go to this link and scroll forward to see the next few slides where we share resources for learning as well as references for the works we presented today. And if you have any questions about Quarto or R Markdown, please stop by the Open Source table in the Lounge between 12-3pm today. And don’t miss the Quarto session at 3:20pm!

La Technique

Installer

Suivre les instructions de Quarto.org.

Remarque

Nécessaire d’avoir une certaine version de l’IDE RStudio (v2022.07).

1. Installer Quarto CLI
2. Installer le package éponyme quarto

Quarto peut s’utiliser en dehors de tout IDE, directement en ligne de commande

Créer un notebook

Une fois l’installation faite, comment lancer Quarto?

• RStudio
• Jupyter

Des formats de fichiers spéciaux proposés dans File: une nouvelle extension, .qmd

Les nouveautés dans le notebook Jupyter: les options globales dans le premier bloc (= YAML header dans .qmd), les options dans les blocs de code (= chunks options .qmd)

Créer un notebook

Une fois l’installation faite, comment lancer Quarto?

• RStudio
• Jupyter

Un bouton Render au lieu de Knitr pour générer le document

Pour prévisualiser le document :

quarto preview hello.ipynb

et pour générer le document:

quarto render hello.ipynb --to pdf
quarto render hello.ipynb --to html

Créer un notebook

Une fois l’installation faite, comment lancer Quarto?

• RStudio
• Jupyter

Une autre manière de spécifier les “chunk options” (pipe natif \(\#|\))

Créer un notebook

Une fois l’installation faite, comment lancer Quarto?

• RStudio
• Jupyter

Deux modes: Source ou Visual, possible d’aller de l’un à l’autre à volonté

Que peut-on faire en Quarto ?

Que peut-on faire en Quarto ?

• des rapports;
• des présentations;
• des sites web;
• des livres et articles.

Quarto rend possible de passer de l’un à l’autre sans (presque) aucune modification dans le code!

Comparaison avec les possibilités en Rmarkdown/Knitr so far

Rapports

• HTML
• PDF
• Word

Quoi de plus que dans Rmarkdown (HTML) ?

Rapport HTML : formats

• margin : Mettre des figures à côté (et non plus juste en dessous) du code

• columns : arranger plus ou moins la place prise par l’objet en fonction du bloc

Rapport HTML: le copier-coller

Possible de copier-coller une image (s’enregistre automatiquement dans un dossier images)

Rapport HTML: le copier-coller

Ainsi que du texte formaté (Quarto va conserver ce format).

Rapport HTML : autres

Les trucs un peu gadget (mais quand même cool!) :

• Pouvoir afficher le rapport en mode clair ou sombre¹.
• Le code: copier-coller, cacher/montrer, séparer automatiquement en plusieurs lignes
• Afficher les citations et notes de bas de page en passant la souris dessus
• code-link ou intégrer l’URL des pages d’aide des fonctions R².

Présentations

• Reveal.js
• Beamer
• Même possible d’exporter en .ppt

Au-revoir les captures écran pour intégrer les graphiques et tableaux depuis R dans les présentations!

Quoi de plus que dans RMarkdown ?

Présentation HTML : surligner le code

Faire apparaître au fur et à mesure le code pour pouvoir le commenter plus facilement

# Most basic bubble plot
data %>%
  arrange(desc(pop)) %>%
  mutate(country = factor(country, country)) %>%
  ggplot(aes(x=gdpPercap, y=lifeExp, size=pop, fill=continent)) +
    geom_point(alpha=0.5, shape=21, color="black") +
    scale_size(range = c(.1, 24), name="Population (M)") +
    scale_fill_viridis(discrete=TRUE, guide=FALSE, option="A") +
    theme_ipsum() +
    theme(legend.position="bottom") +
    ylab("Life Expectancy") +
    xlab("Gdp per Capita") +
    theme(legend.position = "none")

Présentation HTML : chalkboard

• Un outil pour entourer, dessiner, écrire sur le slide
• Un tableau noir + craie avec possibilité d’intégrer les notes aux slides

Pour en savoir plus: chalkboard

Présentation HTML : extension pointeur

$ quarto install extension quarto-ext/pointer

Raccourci = Touche q pour switcher entre le curseur et le pointeur.

Blogs, sites webs

• Démo pour monter un site web en moins de 10 minutes: rstudio::conf(2022) keynote by Mine Çetinkaya-Rundel & Julia Stewart Lowndes

Pas possible ?! Ok, essayons de monter un site web…

1. Créer un compte Quarto.pub³ et se connecter via le navigateur par défaut.
2. Faire le site et le publier.

Blogs, sites webs

• Nouvelle tab qui apparaît (“Build”) pour voir en avant-première le rendu du site (avant publication)
• Une seule ligne dans le terminal pour publier le site: quarto publish
• Possible de monter un site web à la fois avec des outputs R et Python (.qmd et .ipynb)

Articles, books⁴

• On peut spécifier un format connu d’article scientifique (via CLI), i.e. ASS

Quarto est polyglote

Quarto est polyglote

Quarto permet d’exécuter du code

• python,

• R

• Observable.hq

• Julia …

• dans le même notebook

• en partageant les données

les cellules de code

python

import pandas
pingouin = pandas.read_csv("penguins.csv")
# ojs_define( pypingouin = pingouin)

R

library(reticulate)
library(dplyr)
pingouin <- py$pingouin %>% mutate(bill_length_mm = 0.98 * bill_length_mm)
ojs_define( rpingouin = pingouin)

Observable.js

```{ojs}
//| echo: true
render(mySVG)
xProperty = 'flipper_length_mm'
yProperty = 'bill_length_mm'

function render(div) {
  // draw on the SVG with D3
  const selection = d3.select(div).select('svg');
  
  const xExtent = d3.extent(rpingouin, d => d[xProperty]);
  const x = d3.scaleLinear().domain(xExtent).range([30,width-30]);
  const xAxis = d3.axisBottom(x);
  selection.select("#xAxis")
    .call(xAxis);
  
  const yExtent = d3.extent(rpingouin, d => d[yProperty]);
  const y = d3.scaleLinear().domain(yExtent).range([30,height-30]);
  const yAxis = d3.axisLeft(y);
  selection.select("#yAxis")
    .call(yAxis);
  
  const colorScale = d3.scaleOrdinal(d3.schemeCategory10);

  
  selection.selectAll('circle')
    .data(rpingouin.filter(d => d[xProperty] !== "NA"))
    .join('circle')
    .attr('r', 15)
    .style('opacity', 0.2)
    .style('fill', d => colorScale(d.species))
    .transition()
    .duration(3000)
    .attr('cx', d => x(d[xProperty]) )
    .attr('cy', d => y(d[yProperty]) );

  const legend = swatches({color: colorScale});
  d3.select(div).select("#legend")
    .append(() => legend);
  
}

mySVG = html`<div style="width:${width}px; height:${height+50}px">
<div id="legend"></div>

<svg width=${width} height=${height}>
  <circle r=50 cx=100 cy=100 fill="seagreen"></circle>
  <g id="xAxis" transform="translate(0,${height-20})"></g>
  <g id="yAxis" transform="translate(25,0)"></g>
</svg>
</div>

```

OJS Syntax Error (line 46, column 52)Unterminated template

Migration
to Quarto

::: {style=“margin-top: 50px; font-size: 1.5em; color: #75AADB;”} ” Yes but I have so many of the following notebooks already ”

flowchart LR
    B[.Rmd notebook] --fas:fa-magic--> Q(Quarto Notebook)
    A[fa:fa-python .ipynb notebook] --fas:fa-magic--> Q(Quarto Notebook)
    D[Observable.hq notebook] --fas:fa-magic--> Q(Quarto Notebook)
    C[LaTeX] --fas:fa-magic--> Q(Quarto Notebook)
    

Rmd migration
in 2.5 sec

Terminal

mv README.Rmd README.qmd

Rmd migration
in 7.5 sec

Console

# convert to Quarto doc with in-body chunk options
knitr::convert_chunk_header("to-convert.Rmd", "converted.qmd")

New documents :

:::

ipynb migration

• step 1 :

VSCode Command Palette

Jupyter: Import Jupyter Notebook

• step 2 :

VSCode Command Palette

Quarto: New document

Observable.hq migration

• install ohq2quarto from github: Save an Observable HQ Notebook to a Quarto project

• step 1 : use the import command

Terminal

cargo run -- --ohq-ref @hrbrmstr/just-one-more-thing --output-dir ./examples --verbose

LaTeX migration

WiP

(more at the post-event bar)

Et RMarkdown dans tout ça?

• Conversion .Rmd \(\rightarrow\) .qmd simple et lecture de .Rmd via Quarto CLI possible
• Restera à moyen terme supporté par Posit…
• …mais les efforts de développement/nouveautés seront concentrées sur Quarto

Quarto Vs. RMarkdown: les plus

A mettre à la toute fin de la présentation (pas uniquement de cette partie?)

• La plupart des choses peuvent être également faites en Rmarkdown, mais bien plus simple/“élégant” en Quarto (moins de bidouillage peut être)
• Possible de switcher entre de l’édition en code et de l’édition en clique-bouton (type Word, LibreOffice), pratique quand on oublie juste une commande!
• Propositions automatique dans le YAML header (CTRL + espace) et pour les chunk options: pareil, pratique quand on a un trou de mémoire (ou qu’on veut savoir toutes les options disponibles)
  
• Intéragir avec du notebook R ou Python exactement de la même manière
• Facile de naviguer entre plusieurs formats (peu de choses à modifier pour que le document s’édite)
• Gérer les outputs de différentes sortes dans un même document (comme un site web)

Quarto Vs. RMarkdown: les moins

• Tout nouveau, donc peu d’aide dans la communauté disponible

En gros, si ce que l’on veut faire n’est pas décrit dans l’aide du site Quarto, très compliqué de mettre en place l’idée. Fouinage et bidouillage ++++

Merci !

Merci !

Thank you so much for joining us this morning, here in person or from wherever you are tuning in to the live stream. We’ve talked about our experiences using Quarto and all the problems it has helped us solve and reimagine, but while Quarto is stable and ready for use, it’s also a baby that you too can be a part of its growth. So start using Quarto, and get involved by asking questions, reporting issues, providing feedback, and even building extensions for it. We can’t wait to see what you create!

Say hello to Quarto!

• Pick up where we left off: github.com/mine-cetinkaya-rundel/hello-quarto-demo

• rstudio::conf(2022) – Quarto deep dive:

  • Tracy Teal - These are a few of my favorite things (about Quarto presentations)
  • Hamel Husain - Literate Programming With Jupyter Notebooks and Quarto
  • Devin Pastoor - Websites & Books & Blogs, oh my! Creating Rich Content with Quarto

• Video: Reproducible authoring with Quarto

• Video: Openscapes: Hello Quarto with NASA Openscapes, RLadies Santa Barbara, and JJ Allaire

• Quarto documentation: quarto.org

• Quarto gallery: quarto.org/docs/gallery

• Quarto tip a day: rstd.io/quartotip

Annotated Notes: Share

• Hex logos: https://github.com/rstudio/hex-stickers

• How it works: https://quarto.org/docs/get-started/hello/rstudio.html#how-it-works

Annotated Notes: Collaborate

Demo: JupyterLab JupyterHub managed by 2i2c https://2i2c.org

Other Rey with Quarto, throwback to https://jules32.github.io/useR-2019-keynote

Annotated Notes: Teach

• Photo credit for Holland Stam: https://undergraduateresearch.duke.edu/news/data-scraping-art-history-survey-texts

• 📦 arthistory: https://saralemus7.github.io/arthistory

Annotated Notes: Reimagine

FayLab Manual

• Learning by doing: migrating to Quarto from Bookdown

Quarto Mosaic

See also: 3 takeaways for planning for the year of open science

• https://openscapes.github.io/approach-guide
• https://nmfs-openscapes.github.io

Footnotes

1. pas posible pour les slides…

2. problème pour la faire fonctionner en revealjs et html…

3. solution pour publier des sites web, proposée par Quarto (Posit)

4. Probablement d’autres aspects sympas, mais pas d’expérience sur ces formats.