[NetBehaviour] Digital Evaluation Of The Humanities

[Rob Myers]

http://robmyers.org/2011/10/16/digital-evaluation-of-the-humanities/


Humanities Computing dates back to the use of mainframe computers with
museum catalogues in the 1950s. The first essays on Humanities Computing
appeared in academic journals in the 1960s, the first conventions on the
subject (and the Icon programming language) emerged in the 1970s, and
ChArt was founded in the 1980s. But it isn't until the advent of Big
Data in the 2000s and the rebranding of Humanities Computing as the
"Digital Humanities" that it became the subject of moral panic in the
broader humanities.

The literature of this moral panic is an interesting cultural phenomenon
that deserves closer study. The claims that critics from the broader
humanities make against the Digital Humanities fall into two categories.
The first is material and political: the Digital Humanities require and
receive more resources than the broader humanities, and these resources
are often provided by corporate interests that may have a corrupting
influence. The second is effectual and categorical: it's all well and
good making pretty pictures with computers or coming up with some
numbers free of any social context, but the value of the broader
humanities is in the narratives and theories that they produce.

We can use the methods of the Digital Humanities to characterise and
evaluate this literature. Doing so will create a test of the Digital
Humanities that has bearing on the very claims against them by critics
from the broader humanities that this literature contains. I propose a
very specific approach to this evaluation. Rather than using the Digital
Humanities to evaluate the broader humanities claims against it, we
should use these claims to identify key features of the broader
humanities self-image that they use to contrast themselves with the
Digital Humanities and then evaluate the extent to which the literature
of the broader humanities actually embody these features.

This project has five stages:

1. Determine the broader humanities' claims of properties that they
posses in contrast to the Digital Humanities.
2. Identify models or procedures that can be used to evaluate each of
these claims.
3. Identify a corpus or canon of broader humanities texts to evaluate.
3. Evaluate the corpus or canon using the models or procedures.
4. Use the results of these evaluations as direct constraints on a
theory of the broader humanities.

Notes on each stage:

Stage 1

I outlined some of the broader humanities' claims against the Digital
Humanities above that I am familiar with. We can perform a Digital
Humanities analysis of texts critical of the Digital Humanities in order
to test the centrality of these claims to the case against the Digital
Humanities and to identify further claims for evaluation.

Stage 2

There are well defined computational and non-computational models of
narrative, for example. There are also models of theories, and of
knowledge. To the extent that the broader humanities find these
insufficient to describe what they do and regard their use in a Digital
critique as inadequate they will have to explain why they feel this is
so. This will help both to improve such models and to advance the terms
of the debate within the humanities.

One characteristic of broader humanities writing that is outside of the
scope of the stated aims of this project but that I believe is
worthwhile investigating are the extents to which humanities writing is
simply social grooming and ideological normativity within an educational
institutional bureaucracy, which can be evaluated using measures of
similarity, referentiality and distinctiveness.

Stage 3

It is the broader humanities' current self-image (in contrast to its
image of the Digital Humanities) that concerns us, so we should identify
a defensible set of texts for analysis.

There are well established methods for establishing a corpus or canon.
We can take the most read, most cited, most awarded or most recommended
articles established by a particular service or institution from a given
date range (for example 2000-2009 inclusive or the academic year for
2010). We can take a reading list from a leading course on the subject.
Or we can try to locate every article published online within a given
period. Whichever criterion we choose we will need to explicitly
identify and defend it.

Stage 4

Evaluating the corpus or canon will require an iterative process of
preparing data and running software then correcting for flaws in the
software, data, and models or processes. This process should be recorded
publicly online in order to engender trust and gain input. To support
this and to allow recreation of results the software used to evaluate
the corpus or canon, and the resulting data, must be published in a free
and open source manner and maintained in a publicly readable version
control repository.

Stage 5

Stage five is a deceptive moment of jouissance for the broader
humanities. It percolates number and model into narrative and theory,
but in doing so it provides a test of the broader humanities' self-image.

For the broader humanities to criticise the results of the project will
require its critics to understand more of the Digital Humanities and of
their own position than they currently do. Therefore even if the project
fails to demonstrate or persuade it will succeed in advancing the terms
of the debate.