Data engineer
SRJC Course Outline Archive
Preserved 23,760 course-version records and 23,760 source PDFs outside a legacy portal.
This was delivered work, not a public site, so the proof here is real screenshots of what shipped plus the write-up below. Happy to walk through the actual deliverable on a call.
Santa Rosa Junior College course-outline records lived in a legacy ASP.NET WebForms portal with no application programming interface or bulk export. We built a custom extraction and normalization pipeline, reconciled 23,760 course-version records, and preserved the corresponding 23,760 source PDFs in a structured archive prepared for research, preservation, and migration planning.
The problem
Decades of public course-outline data were available only through a legacy ASP.NET WebForms portal with no application programming interface, bulk export, or clean route into another system. The archive needed to preserve individual versions, source documents, identifiers, and relationships without silently dropping records.
What we built
We built a custom integration layer that issued more than 3,000 stateful queries, parsed the returned pages, normalized course and version metadata, reconstructed course lineage, and downloaded the source PDFs. Record counts and downloads were reconciled so the resulting archive could support research, preservation, and migration planning.
What was delivered
- Structured dataset covering 23,760 course-version records across 2,382 course families
- Archive of 23,760 source PDFs spanning Fall 1981 through Fall 2025
- Normalized course, version, term, status, and source-document fields
- Course-lineage relationships and record-reconciliation reports
- Migration-ready exports for mapping into a target curriculum or institutional system
Outcomes
- 23,760 of 23,760 identified source PDFs downloaded successfully
- Independent archive no longer dependent on one legacy search interface
- Structured records suitable for analysis, preservation, and target-system mapping
- A reusable extraction and verification pattern for other institutional datasets
Services: Legacy-system extraction, Data normalization, Record reconciliation, PDF archive, Migration-ready exports