Challenges of Digital Archives at UAlbany - Greg Wiedeman

Challenges of Digital Archives at UAlbany

Gregory Wiedeman
University Archivist

The University Archives

University Records
- Office of the President
- University Senate
- University Council
- Office of the Provost
- Graduate & Undergraduate Education
  - Records of Schools and Colleges
  - Records of Academic Departments
- Web Archives
Student Groups and Manuscripts
- Student Association
- Albany Student Press
- Faculty and Alumni Papers

SUNY Records Retention

University records are public records, subject to FOIL

Collecting Background

Formal records management program until cut in 1990s
- Very effective collecting in 1960s and 1970s
Records Management “distributed” among offices
Permanent Records should come to me
Transfers used to happen during moves, when space needed
- No longer sufficient
Misunderstanding of archives and our role

Building a Modern Collecting Program

Automated digital transfers
Web crawling
Disk imaging of external media
Paper when maintained as such
Format-neutral practices
Requires digital infrastructure

Born-Digital Records Collecting in Practice

Files On the Live Web
- Web crawling
Files dispersed around the University
- Local Computers
- Network Shares
- Cloud Storage
- Different Software Applications
- Private Computers

Ben Goldman, “Outfitting a Born-Digital Archives Program”
Practical Technology for Archives Issue 2 (June 2014)

Transfer Scripts

Network Folder Share
- Archives and creator has access
Python script run on task scheduler
Weekly checks for new files
CSV log files of files transferred
Creates XML accession metadata file
Runs createSIP.py command line tool
Enters line in accessions.xlsx
Manual collecting over GoogleDrive

Bag-it

What our bags currently look like

Maintenance

Scripts break over time
- No more ad-hoc approaches
- No more XML data stores
Network of well-maintained interoperable tools

Building Digital Infrastructure

Implementing Maintainable tools

Connections through APIs

Archive-It API
Wayback Machine API
ArchivesSpace
ArchivesSpace API

python from archives_tools import aspace as AS session = AS.getSession() for collection in AS.getResources(session, “2”, “all”): if “ua” in collection.id_0: print collection.title

for collection in AS.getResources(session, “2”, “all”): if “Baldus” in collection.title: for note in collection.notes: if note.type == “abstract”: AS.pp(note)

Espy Project: Building an Open Digital Repository

CLIR-funded grant project to digitize Espy Papers
Make data publicly available for computational research
Espy Metadata Creation Tool

Portland Common Data Model

Example Data Model

Open, Interconnected Workflows

Detect accession
Get contextual data from ArchivesSpace API
Ingest into digital repository
Que for review
Expose data into central search index

Central Search index

Other Projects

Questions