[NetBehaviour] CAT is Decommissioning this Weekend (fwd)

[Alan Sondheim]

Might be of interest here, Alan


---------- Forwarded message ----------
Date: Wed, 9 Sep 2020 07:00:56
From: noreply at texifter.com
To: noreply at texifter.com
Subject: CAT is Decommissioning this Weekend

It is with serious regrets that I need to announce we will decommission the Coding Analysis Toolkit this coming weekend. Over time, our ability maintain the software, now 13 years old, has been impacted by life events and career changes.

A few quick notes:

- The code is open source; someone else could host a version.
- The users did amazing, diverse, and important work.

https://discovertext.com/2018/03/31/scholarly-citations-of-the-coding-analysis-toolkit/

The service grew out of a need to enable groups to accurately classify text and more easily measure and report the results. We learned a lot about humans and software along the way. Some of the ten insights gleaned from my 20 years of using software to label text data (listed below) will, I hope, stand the test of time:

- Some labeling/coding tasks are harder than others.
- Some humans are better than others.
- Average humans are often not nearly good enough.
- Using and comparing just two coders is problematic.
- Find domain specific ways to rank humans over time.
- Even the best human coders struggle with some tasks.
- CoderRank weighted machine-learning (ML) training sets.
- When training machines (ML), start with easier tasks.
- Easier means fewer codes at once where a binary is best.
- Easier means "relevance" first to clean messy data.

I want to thank all the users who patiently sent feedback in the early years. This platform could never have existed without the amazing dedication of solo engineer and software maestro Mark J. Hoy. His vision for what I wanted to see in software and his ability to execute on it never once let me down.

It has been a journey...but we are signing off.

Thanks,
~Stu

-- 
Dr. Stuart W. Shulman
Founder and CEO, Texifter
Editor Emeritus, Journal of Information Technology & Politics