Codnut

Textbook Data Processing and Extraction System

We prepared primary and secondary school textbook data for an AI tutor, then improved and extended the system that extracts it.

Oct 2023 – May 2025

  • #Data Processing
  • #Backend
  • #Automation
  • #AI Data

Project

  • Oct 2023 – Mar 2024 — processed secondary school textbook data.
  • Sep 2024 – May 2025 — improved and extended the textbook data extraction system used to build an AI tutor.

Challenge

  • Textbook layout is complex, so extracting usable training data requires a dedicated processing step.
  • Each subject structures its book differently, so one rule does not cover them all.
  • If extraction is inaccurate, the quality of the AI-generated questions built on top of it drops with it.

Development

Textbook data preparation

  • Extracted text from secondary school textbook PDFs across several publishers and loaded it into a database.
  • Classified and tagged units, page numbers, learning objectives and body text.
  • Split the working method by subject and by how each book was structured.
  • Ten local crowd workers carried out the work against the processing standard.

Extraction system improvements

  • Improved and extended the extraction system across primary and secondary textbooks.
  • Replaced manual steps with automated extraction.
  • After the improvements, the accuracy of the AI-generated questions exceeded the target.

Technology

  • Python
  • PDF Parsing
  • Data Labeling
  • Backend

Related links

Other projects

← Back to Works