Course data

The shared playground

One dataset, all semester: the 303 characters in Wikipedia's Category:Marvel Comics superheroes — first the network of links between their pages, later the pages themselves. Released in frozen stages that follow the syllabus, so every group computes on identical data while Wikipedia moves on. Week 4 is the one exception: communities need a network with lots of them, so that week runs on a second snapshot, the philosophers born before 1900.

Releases

Files	Arrives	What
week1_edges.tsv week1_nodes.tsv	week 1 · out now	The unweighted directed network: an edge A → B wherever A's article links to B's. 303 nodes, 1,784 edges. The node file carries names, Wikidata ids, URLs and one-line blurbs — no degrees, no statistics: computing those is your job.
week4_edges_weighted.tsv	week 4 · out now	The same 1,784 edges with a weight: how many times A's article links to B's. Most links happen once; a few characters link to each other a dozen times. Sum the two directions for an undirected weighted network. Same node file as week 1.
week4_philosophers_edges.tsv week4_philosophers_nodes.tsv	week 4 · out now	Week 4's own network: philosophers. Everyone on Wikipedia's seven lists of philosophers born before the 20th century (List of philosophers born in the centuries BC … born in the 19th century): 1,444 nodes, 11,135 directed edges, each with a weight (how many times A's article links to B's). The node file adds two list-membership columns: `era` (which century list) and `subfields` (any of Wikipedia's subfield lists, e.g. ethicists or logicians, the philosopher is also on).
marvel_pages.zip	week 5 · out now	The text half of the same playground. The full plain-text Wikipedia article for every one of the 303 characters, one file per character — the same 303 nodes as `week1_nodes.tsv`, so the language joins straight onto the network you already have. 1.8 MB zipped, 4.4M characters of prose. Pages run from 1,244 characters to 87,256, median 7,642 — a 70× spread that is worth looking at before you average anything across documents.

Loading it

The network files are tab-separated with # comment headers; node ids are Wikipedia page titles with underscores. One thing to know: 17 of the 303 characters have no link in either direction, so if you build the graph from the edge list alone you get 286 nodes and silently lose the isolates — and the isolates are part of the story. Add every node first:

import pandas as pd, networkx as nx

nodes = pd.read_csv("week1_nodes.tsv", sep="\t", comment="#",
                    quoting=3)  # QUOTE_NONE: TSVs never quote, but blurbs may contain "quotes"
edges = pd.read_csv("week1_edges.tsv", sep="\t", comment="#",
                    names=["source", "target"])

G = nx.DiGraph()
G.add_nodes_from(nodes.node_id)
G.add_edges_from(edges.itertuples(index=False))
# G: 303 nodes, 1784 edges

The philosophers network (week 4) loads the same way; sum the two directions to get the undirected weighted network the week works with:

nodes = pd.read_csv("week4_philosophers_nodes.tsv", sep="\t", comment="#", quoting=3)
edges = pd.read_csv("week4_philosophers_edges.tsv", sep="\t", comment="#")

G = nx.Graph()
G.add_nodes_from(nodes.node_id)
for s, t, w in edges.itertuples(index=False):
    if G.has_edge(s, t):
        G[s][t]["weight"] += w
    else:
        G.add_edge(s, t, weight=w)
giant = G.subgraph(max(nx.connected_components(G), key=len)).copy()
# G: 1444 nodes, 9140 links; giant: 1374 nodes, 9139 links

The text ships as one zip so it is a single download. Unzipping is optional — Python reads it in place, and decoding the filenames gives you keys that are exactly the node_ids from the network files:

import zipfile, urllib.parse

with zipfile.ZipFile("marvel_pages.zip") as z:
    marvel_pages = {
        urllib.parse.unquote(n.split("/")[-1][:-4]): z.read(n).decode("utf-8")
        for n in z.namelist()
        if n.endswith(".txt") and "README" not in n
    }
# marvel_pages: 303 pages, keyed by node_id — set(marvel_pages) == set(nodes.node_id)

Why the decoding step: a filename cannot contain every character a Wikipedia title can, so the titles are URL-encoded on disk — Mark_Hazzard%3A_Merc is Mark_Hazzard:_Merc. Skip the unquote and that one page will quietly fail to join against the network.

Where it comes from

The Marvel snapshot was built 26 August 2026 from the live English Wikipedia (the philosophers on 15 September, with the same pipeline) — network and text come from the same crawl on the same day, so they describe the same 303 articles. Three decisions in the pipeline are worth knowing about, because they are exactly the kind of decision §3 of week 1 is on about:

Edges come from the article text, not from Wikipedia's link index. The index counts links contributed by shared navigation templates — every Avenger carries the same team navbox, which would make whole rosters mutually adjacent by templating artefact, not because anyone's article mentions anyone. Here an edge means the running text of A's article actually links B.
Redirects are resolved on both ends, so [[Spidey]] and [[Spider-Man]] are one node, and a link to any alias still lands on the character.
The text is the rendered prose, not the wiki markup. Templates, infoboxes, reference markers and navigation furniture are gone; what is left is what a reader sees. That is a preprocessing decision made for you, and week 5 §2 is about exactly this kind of decision — so it is fair game to ask what it removed. The section headings are still in there, and so is a good deal of Wikipedia's house phrasing.

Could you have made those choices differently? Yes — and you would have gotten a different network. That is the point of week 1, and when your project crawls a domain of its own, the choices are yours.

← All weeks