Skip to content

Commit 9cfee60

Browse files
author
NullConvergence
committed
Added pubs + ranking + report pages and tweaked text
1 parent 4eb0900 commit 9cfee60

29 files changed

+445
-52
lines changed

‎_config.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,11 +72,13 @@ collections:
7272
permalink: /news/:path/
7373
staff:
7474
people: true
75+
pub:
76+
people: true
7577
references:
7678
people: true
7779

7880

79-
news_limit: 5
81+
news_limit: 3
8082

8183
# -----------------------------------------------------------------------------
8284
# Jekyll settings

‎_includes/header.html

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,22 +23,26 @@
2323
<!-- Practices -->
2424
<a class="page-link" href="{{ '/practices' | prepend: site.baseurl | prepend: site.url }}/">practices</a>
2525

26-
<!-- About -->
27-
<a class="page-link" href="{{ site.baseurl | prepend: site.url }}/">about</a>
26+
<!-- Publications -->
27+
<a class="page-link" href="{{ '/publications' | prepend: site.baseurl | prepend: site.url }}/">publications</a>
28+
2829

2930
<!-- Blog -->
3031
<a class="page-link" href="{{ '/blog/' | prepend: site.baseurl | prepend: site.url }}">blog</a>
3132

33+
<!-- About -->
34+
<a class="page-link" href="{{ site.baseurl | prepend: site.url }}/">about</a>
35+
3236
<!-- Members -->
33-
<a class="page-link" href="{{ '/members/' | prepend: site.baseurl | prepend: site.url }}">members</a>
37+
<a class="page-link" href="{{ '/members/' | prepend: site.baseurl | prepend: site.url }}">team</a>
3438

3539

3640
<!-- Pages -->
37-
{% for page in site.pages %}
41+
{% comment %} {% for page in site.pages %}
3842
{% if page.title and page.url != "/" %}
3943
<a class="page-link" href="{{ page.url | prepend: site.baseurl | prepend: site.url }}">{{ page.title }}</a>
4044
{% endif %}
41-
{% endfor %}
45+
{% endfor %} {% endcomment %}
4246

4347
</div>
4448
</nav>

‎_includes/news.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
<h2>news</h2>
33
{% if site.news %}
44
<ul>
5-
{% assign news = site.news %}
5+
{% assign news = (site.news | sort: 'date') | reverse %}
66
{% for item in news limit: site.news_limit %}
77
<li>
88
<p>{{ item.content | remove: '<p>' | remove: '</p>' | emojify }} </p>

‎_layouts/practice.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ <h1 class="post-title">{{ page.name }}</h1>
4848
<a href="{{last_pr[0].url}}" class="float-left"><i class="fa fa-arrow-left" aria-hidden="true"></i></a>
4949
<span> {{page.index }} / {{sorted_practices.size}} </span> •
5050
<a href="/practices#{{page.category | downcase}}"> <span>{{page.category}}</span> </a> •
51-
<a class="badge parent" href="/blog/2020/catalog/">
51+
<a class="badge parent" href="/ranking">
5252
<img class="hover-me" src="{{badge}}"></img>
5353
<span class="show-me">
5454
<span class="show-me-body">{{badge_text}}</span>

‎_news/catalog_live.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
---
22
layout: post
3-
date: 2020-01-15 14:50:00-0400
3+
date: 2020-01-25 14:50:00-0400
44
meta: 1
55
inline: true
66
---
77

8-
We launched a **catalog** of Software Engineering best practices for Machine Learning applications. Read more about it <a href="/blog/2020/catalog/">here</a>!
8+
We launched a **catalog** of Engineering best practices for Machine Learning applications. Read more about it <a href="/blog/2020/catalog/">here</a>!

‎_news/esem_paper.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
layout: post
3+
date: 2020-07-15 00:00:00-0400
4+
meta: 1
5+
inline: true
6+
---
7+
8+
Our paper "Adoption and Effects of Software Engineering Best Practices in Machine Learning" was accepted at ESEM 2020.

‎_news/report.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
layout: post
3+
date: 2020-07-16 00:00:00-0400
4+
meta: 1
5+
inline: true
6+
---
7+
8+
We published a report on the "2020 State of Engineering Practices for Machine Learning". Read more about it <a href="/report2020">here</a>!

‎_news/we_live.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
layout: post
3-
date: 2020-01-29 14:50:00-0400
3+
date: 2020-01-10 14:50:00-0400
44
meta: 1
55
inline: true
66
---

‎_pages/about.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,9 @@ news: true
1313
social: true
1414
---
1515

16-
An ever-increasing number of organisations are developing applications that involve machine learning components. The complexity and diversity of these applications calls for engineering techniques to ensure that they are built in a **robust** and **future-proof** manner.
16+
An ever-increasing number of organisations are developing applications that involve machine learning components. The complexity and diversity of these applications calls for engineering techniques to ensure these are built in a **robust** and **future-proof** manner.
1717

1818
On this website we collect, validate and share engineering best practices for software including ML components.
1919
To this end, <a href="/members">we</a> study the scientific and popular literature and engage with machine learning practitioners.
2020

21+
For more information access our <a href="/practices">catalog of ML engineering best practices</a> or read our annual report on the <a href="">State of Engineering Practices for Machine Learning</a>.

‎_pages/members.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
layout: post
3+
title:
4+
comments: False
5+
permalink: /members
6+
---
7+
8+
<div>
9+
<img src="/assets/img/team.png" class="blog-image">
10+
</div>
11+
12+
<br><br>
13+
{% assign sorted_staff = site.staff | sort:"index" %}
14+
More information:
15+
<ul>
16+
{% for st in sorted_staff %}
17+
<li>
18+
<a href="{{st.webpage}}" target="_blank">{{ st.first_name }} {{ st.last_name }}</a>
19+
</li>
20+
{% endfor %}
21+
</ul>

‎practices/index.html renamed to ‎_pages/practices.md

Lines changed: 5 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,21 @@
11
---
22
layout: default
3+
permalink: /practices
34
---
45

5-
<h1 class="black"> ML Engineering Best Practices </h1>
6+
<h1 class="black"> Engineering best practices for Machine Learning </h1>
67

78

89
<div class="large-image">
9-
<img src="/assets/img/SE4ML-practices-diagram.png" class="blog-image"></img>
10+
<img src="/assets/img/SE4ML-practices-diagram.png" class="blog-image">
1011
</div>
1112
<br>
1213
The list below gathers a set of engineering best practices for
13-
developing software systems with machine learning components.
14+
developing software systems with machine learning (ML) components.
1415
<br><br>
1516
These practices were identified by engaging with ML engineering teams and
1617
reviewing relevant <a href="https://github.com/SE-ML/awesome-seml/blob/master/readme.md" target="blank"> academic and grey literature</a>.
17-
We are running a <a href="https://se-ml.github.io/survey/" target="blank"> global survey</a>
18+
We are constantly running a <a href="https://se-ml.github.io/survey/" target="blank"> global survey</a>
1819
among ML engineering teams to measure the adoption of these practices.
1920
<br><br>
2021
The various practices are grouped into 6 categories, as illustrated in the diagram above.
@@ -28,8 +29,6 @@ <h3 class="black bold" id="data"> Data </h3>
2829
"Whoever owns the data pipeline will own the production pipeline for machine learning."
2930
--<a href="https://twitter.com/chipro/status/1232105910631157761">Chip Huyen<a>
3031

31-
<br><br>
32-
3332
<ul>
3433
{% for pr in sorted_practices %}
3534
{% if pr.category == 'Data' %}
@@ -47,8 +46,6 @@ <h3 class="black bold" id="experiment"> Training </h3>
4746

4847
"No amount of experimentation can ever prove me right; a single experiment can prove me wrong." --Albert Einstein
4948

50-
<br><br>
51-
5249
<ul>
5350
{% for pr in sorted_practices %}
5451
{% if pr.category == 'Experiment' %}
@@ -67,8 +64,6 @@ <h3 class="black bold" id="coding"> Coding </h3>
6764
"You can’t be an AI expert these days and not have some grounding in software engineering."
6865
--<a href="https://twitter.com/Grady_Booch/status/1004078706518736896">Grady Booch</a>
6966

70-
<br><br>
71-
7267
<ul>
7368
{% for pr in sorted_practices %}
7469
{% if pr.category == 'Coding' %}
@@ -87,8 +82,6 @@ <h3 class="black bold" id="deployment"> Deployment </h3>
8782
“If your model isn’t deployed into production, does it really exist?”
8883
--<a href="https://twitter.com/code_star/status/1041913975154331648">Anonymous<a>
8984

90-
<br><br>
91-
9285
<ul>
9386
{% for pr in sorted_practices %}
9487
{% if pr.category == 'Deployment' %}
@@ -105,8 +98,6 @@ <h3 class="black bold" id="team"> Team </h3>
10598

10699
"If you want to go fast, go alone; but if you want to go far, go together." -- African proverb, allegedly
107100

108-
<br><br>
109-
110101
<ul>
111102
{% for pr in sorted_practices %}
112103
{% if pr.category == 'Team' %}
@@ -124,8 +115,6 @@ <h3 class="black bold" id="governance"> Governance </h3>
124115
"Where there is great power there is great responsibility."
125116
--<a href="https://quoteinvestigator.com/2015/07/23/great-power/">Winston Churchill<a>
126117

127-
<br><br>
128-
129118
<ul>
130119
{% for pr in sorted_practices %}
131120
{% if pr.category == 'Governance' %}

‎_pages/publications.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
layout: default
3+
permalink: /publications
4+
---
5+
6+
<h3 class="black"> Our publications: </h3>
7+
8+
9+
<ul>
10+
{% assign sorted_pub = (site.pub | sort: 'date') | reverse %}
11+
{% for item in sorted_pub %}
12+
<li>
13+
<a href="{{item.webpage}}">{{item.title}}</a>
14+
</li>
15+
{% endfor %}
16+
</ul>

‎_pages/ranking.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
layout: post
3+
title: Ranking the Practices by Adoption
4+
comments: False
5+
permalink: /ranking
6+
---
7+
8+
Using the results from our <a href="/survey">survey</a>, we ranked the practices based on three level of difficulty: basic, medium and advanced.
9+
10+
The ranking algorithm computes, for each practice, the percentage of "Completely" answers (corresponding to at least high adoption of a practice), the percentage of "Completely + Mostly" answers (corresponding to at least medium adoption of a practice) and the percentage of "Completely + Mostly + Partially" answers (corresponding to at least low adoption of a practice).
11+
12+
The final rank is the average of: the practice rank on at least high, its rank on at least medium, and its rank on at least low adoption.
13+
By accumulating the answers in this manner, we expect to cancel out the noise stemming from fuzzy boundaries between subsequent answer types or the relative unachievability of complete adoption of a particular practice.
14+
15+
We split the list of ranked practice in 3 percentiles corresponding to "basic" (first group with high adoption), "medium" (second group) and "advanced" (third group with low adoption).
16+
17+
An overview of the percentage of answers for each practice is illustrated below.
18+
19+
<div>
20+
<img src="/assets/img/ranked_practices.png" class="blog-image">
21+
</div>

‎_pages/report.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
layout: post
3+
title: 2020 State of Engineering Practices for Machine Learning
4+
comments: False
5+
permalink: /report2020
6+
---
7+
8+
Report data

‎survey/index.html renamed to ‎_pages/survey.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
---
2-
layout: default
2+
layout: post
3+
title: Survey on Engineering Best Practices for Machine Learning
4+
comments: False
5+
permalink: /survey
36
---
47

5-
<h2 style="color: black;">Survey on Software Engineering Best Practices for Machine Learning</h2>
6-
<br>
78

89
We are running an online survey to measure the adoption of software engineering practices by teams that develop applications with machine learning components.
910

‎_pub/2020_report.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
title: 2020 State of Engineering Practices for Machine Learning
3+
webpage: /report2020
4+
description: #
5+
date: 2020-07-21 14:50:00-0400
6+
---

‎_pub/awesome_list.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
title: Awesome Software Engineering for Machine Learning
3+
webpage: https://github.com/SE-ML/awesome-seml #
4+
description: #
5+
date: 2020-01-10 14:50:00-0400
6+
---

‎_pub/esem_paper.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
title: Adoption and Effects of Software Engineering Best Practices in Machine Learning
3+
webpage: https://github.com/SE-ML/awesome-seml #
4+
description: #
5+
date: 2020-07-20
6+
---

‎_staff/aser.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
first_name: Alex
33
last_name: Serban
4+
index: 1
45
webpage: https://cs.ru.nl/~aserban #
56
---

‎_staff/hhos.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
first_name: Holger
33
last_name: Hoos
4+
index: 3
45
webpage: http://ada.liacs.nl/members/ #
56
---

‎_staff/jvsr.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
first_name: Joost
33
last_name: Visser
4+
index: 4
45
webpage: https://jstvssr.github.io #
56
---

‎_staff/kvdb.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
first_name: Koen
33
last_name: van der Blom
4+
index: 2
45
webpage: http://liacs.leidenuniv.nl/~blomkvander/ #
56
---

‎articles/_posts/2020-01-30-survey.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,12 @@ layout: post
33
title: A global survey on SE4ML
44
date: 2020-01-28 15:32:00-0400
55
comments: True
6-
description: Measuring the adoption of software engineering best practices for machine learning.
6+
description: Measuring the adoption of engineering best practices for machine learning.
77
image: /assets/img/awesome_diagram.png
88
photocredit: #
99
---
1010

11-
We have reviewed both scientific literature and popular publications to identify about **40** software engineering best practices that were recommended or used by teams that develop machine learning applications.
11+
We have reviewed both scientific literature and popular publications to identify **29** engineering best practices that were recommended or used by teams that develop machine learning applications.
1212

1313
Based on the identified practices, we created a **7**-minute online survey that asks respondents to what extent they are adopting those practices.
1414

‎articles/_posts/2020-02-26-catalog.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
---
22
layout: post
3-
title: A catalog of Software Engineering best practices for Machine Learning
3+
title: A catalog of Engineering best practices for Machine Learning
44
date: 2020-02-26 15:32:00-0400
55
comments: True
6-
description: We published a catalog of software engineering best practices for ML applications.
6+
description: We published a catalog of engineering best practices for ML applications.
77
image: /assets/img/SE4ML-practices-diagram.png"
88
photocredit: #
99
---

‎assets/img/ranked_practices.png

1.03 MB
Loading

‎assets/img/team.png

1.93 MB
Loading

‎members/index.html

Lines changed: 0 additions & 17 deletions
This file was deleted.

0 commit comments

Comments
 (0)