update on the ever-changing workflow

Summer is that time when I get the most grunt work done on research projects. And when I’m immersed in it, I can’t help but tweak around with my workflow. I did a bunch of tweaking last summer, and am gearing up for some more in just a month and a half! Prompted by recent posts by Shane Landrum and Bill Turkel, and as an update on these posts, I thought it would be worthwhile to write up the ever-changing nature of my current workflow, the apps that make it possible, and how it works.

The changes reflect a combination of factors, including my incorporation of new applications/services (dropbox, for example), updates to apps I’ve used over the past four years (DEVONthink), and new ambitions for my research files (web publication, textual analysis, etc.).

What values are driving my workflow?

• Platform independence, and future-proofing. I started my graduate career in 1997 as an avid devotee of WordPerfect, which I had been using at least since the release of WordPerfect 5.1 in 1989. When I finished my masters thesis in 1999, I had to convert the final product to Word in order to please my department and the ruler nazis in charge of enforcing UMI standards. It was a total pain, as formatting and footnotes took a hit in the transformation. This was the first in many lessons I had in the value of platform independence and future proofing one’s notes. There have been many lessons since then, and I’ve finally started listening. Now, I’m committed to keeping as much of my work as possible in the most durable of file formats, plaintext files. TXT files are about as platform independent and future proof as one can find, though not completely.
• Flexible files. I’m transcribing an absolute ton of documents from [archival photos]. What is more, I need those transcriptions to do a variety of things. I need them for text analysis using Python’s nlp libraries (nltk and orange). I need the transcriptions, together with images of the manuscripts, for upload to the project archive site. I also want to extract specific information, such as names, crimes, judges, etc., from specific documents. This aspect of my work— which increasingly uses computational aids for textual analysis— requires accessibility and flexibility of my files.
• Robust searching. I still want the kind of robust full-text searching that drew me to DEVONthink originally. Note, though, that the old way I was using DEVONthink violates both platform independence and flexible access. (This is true, I would argue, even though DT now saves files outside the application and in the file system. The file tree is driven by the needs of the app, rather than by accessibility outside the app.) I’m no longer married to Macs, and even though I have a bunch of them, my next computer will likely be powered by linux.
• Ubiquitous access and backup. Yes, I do have too many Macs— an iMac at home in New Mexico, and one at work in Tennessee, a 13in MBP I travel with, and a 15in MBP at work, an old black MacBook I got when I started at UTK. Plus, I have my eyes on one of these, with Ubuntu 10.10 on it. Oh, and there’s always the iPhone pocket computer too. I want easy access to my files with whichever computer I happen to be using at any moment. I also want redundant backups and version control for the different manifestations of my transcriptions. On the redundant backup front, I use superduper to keep a bootable copy of my main laptop in an exterior hard drive. I also use backblaze for cloud backup.

What am I using?

1. Textmate for transcribing, note-taking, blog writing, and general project organization. Indispensable to this is the missingdrawer plugin, which allows me to add new folders and files to the file structure from within Textmate. I use markdown for much of my note-taking, transcribing, and writing. I also code Python scripts in Textmate.
2. oXygen for XML authoring of TEI versions of my transcriptions. If you do any TEI work, oXygen is the standard-setting platform and includes all kinds of features.
3. Subversion for version control. I have svn repositories hosted on my web server for transcriptions, secondary source note-taking, article manuscripts, syllabi, my CV, blog posts, etc. To manage the projects, I use this plugin for the finder, Textmate’s svn package, or oxygen’s SVNClient. I have working copies checked out on all my machines. For those not up for serious version control, ForeverSave works very well locally on your machine.
4. Dropbox for syncing across computers. I have all of the research for my next project in a folder on my main machine that is symlinked to my dropbox. As a result, I have access to all of those files on each of my machines. I keep PDFs of articles, of primary-source books downloaded from Google books here. If they need it, I OCR the files with Adobe Acrobat 9. I checkout the appropriate svn repos/projects here for versioned files related to my project.
5. Notational Velocity and Simplenote synced together for quick-idea note-taking. Notational Velocity allows you to choose between storing your notes in a database or as plaintext files in the file tree. I do the latter, and index that folder with DT. On my iPhone I can make notes with Simplenote while on the go— usually in an airplane!
6. DEVONthink for indexing and searching my research collection. DT’s search capabilities can’t be beat. On each of my machines I have an indexed database instance that indexes my dropbox research folder, and the local notational velocity folder. I fire it up when I’m ready to do analysis and organizing of sources to write with.
7. Zotero Standalone for citation management. Zotero is the best at scraping the web.
8. Scrivener for serious writing. Scrivener 2.0 now allows for export/backup/syncing of files. It also works with markdown/multimarkdown. My use of Scrivener hasn’t changed that much from the old days— I drag it all into the research folder when it’s time to write seriously, and use the split screen. Scrivener is now being developed for Windows and Linux. I find this very exciting.
9. Terminal.app for all kinds of magic.

Other apps that show up from time to time:

Skim and Preview.app for PDF reading/highlighting
Lightzone and Preview.app for displaying jpegs of my manuscripts.
Acorn for some types of image editing.
FileMaker Pro 11 for managing a database of criminals. This one was bought for my by my department. Ultimately, I’ll export that database into a RDBM for deployment.

That’s enough for now. In the coming weeks I’ll write up some specific examples with pictures and everything!

About ctb

Associate Professor of Early Latin America Department of History University of Tennessee-Knoxville

‹ batch renaming archive photos

SECOLAS 2011 ›

Tagged with: method
Posted in Research and Writing