GovEd e-Learning System (Updates In Progress)

Purpose

Academic

Learning Centered Design Project

Overview

Problem Statement

Legislation at the local, state, and federal levels affects users' rights, resources, and daily lives. The internet makes it easy to find information without a clear way to vet its context, accuracy, and relevance. This pain point can stem from a lack of foundational understanding of our government's structure and how it functions. The gap between both access and understanding presents an opportunity to design an e-learning system that builds users' civic knowledge of the three branches of government and empowers them to better evaluate and understand legislation and its impact.

Scope

This iteration of the project focuses on developing an e-learning course that teaches users about the City of Chicago government.

Context of Use

The prototype is an online, self-paced e-learning course, accessible via desktop, tablet, or mobile device.

Course Elements
  • Pretest
  • Pretraining
  • Posttest
Target Users
  • Adults who live, work, or attend school in the City of Chicago and want to understand how Chicago government works
  • High school students who live or attend school in the City of Chicago and want to build their civic literacy
Goals

The e-learning course seeks to help learners achieve the following five objectives according to the Revised Bloom's Taxonomy Table (RBT).

  • Learning Objective 1: Remember (1.2 Recall) knowledge of civic education concepts and terminology(Aa Factual Knowledge)
  • Learning Objective 2: Remember (1.2 Recall) knowledge of government offices and responsibilities (Ba Conceptual Knowledge)
  • Learning Objective 3: Understand (2.6 Compare) different government offices and responsibilities (Ba Conceptual Knowledge)
  • Learning Objective 4: Analyze (4.1 Differentiate) criteria to identify appropriate government offices and obtain information or complete a process (Cc Procedural Knowledge)
  • Learning Objective 5: Apply (3.1 Carry Out) knowledge of government offices and responsibilities (Db Meta-Cognitive)
Principles

This project applies the following seven principles from e-Learning and the Science of Instruction by Ruth Colvin Clark and Richard E. Mayer.

  • Multimedia Principle
  • Contiguity Principle
  • Pretraining Principle
  • Segmenting Principle
  • Personalization Principle
  • Learner Control Principle
  • Worked Examples Principle

Research Method

Pretest-Posttest Design

Project Challenges

The project used a Qualtrics account that is linked to a DePaul University email that is no longer retrievable.

  • Participant data cannot be assessed and data is limited to project notes and reports.

The project did not include a questionnaire to capture participant demographics.

  • The limited project documentation does not include a complete list of the twelve participants who received the Qualtrics study link.
  • The project documentation identifies the age range and education for nine of the twelve participants, but not their residency. (See the Instructional Design section)

This project cannot confirm whether the seven participants who completed the course meet the target user criteria.

  • The project documentation does not clearly link the seven participants to the nine recorded participants.
  • Of the nine recorded participants, the high school-level and Bachelor's-level participants do not live or attend school in the City of Chicago.

The pretest and posttest assessment collected limited objective-level granularity.

  • Pretest and posttest scores were collected in aggregate rather than broken out by individual learning objective. The data confirms the course worked overall (d = 1.14), but it cannot pinpoint which specific objective(s) drove that gain, or, in Participant 4's case, which objective(s) were not met.

Project Value From a UX Perspective

The course flows are functional.

  • Seven participants successfully completed the pretest, pretraining, and posttest.

The project goals span multiple levels of RBT.

  • The assessment didn't just test memorization, it tested whether learners could reason with and act on the material.

The data indicates an overall gain of d = 1.14.

  • The instructional design scaffolded effectively from basic terminology toward applied understanding.

Project Value From a Project Management Perspective

The course provides a defensible business case.

  • The Data Analysis section demonstrates statistically significant, quantifiable evidence of training effectiveness. This initial value justifies continued investment and iterations.

The research establishes a baseline for iteration.

  • The before and after benchmarks mean future courses can be measured against a consistent standard, turning a one-time validation into a repeatable evaluation process.

The research reduces the risk in decision-making.

  • Statistically grounded evidence provides a clear, actionable signal for go/no-go decisions.

Process

Instructional Design

This study was designed according to seven e-learning principles from the Clark and Brown framework describe above to meet the five Bloom's Taxonomy learning objectives (described above).

Lorem Ipsum

Question Learning Objective
1. What is your name N/A
2. What does the executive branch of the government do? LO1, LO5
3. What does the legislative branch of the government do? LO1, LO5
4. What does the judicial branch of the government do? LO1, LO5
5. Which of the following are city-wide elected positions in the City of Chicago? LO2, LO5
6. What does the mayor's office do? LO2, LO5
7. What does the City Clerk do? LO2, LO5
8. What does the Treasurer's Office do? LO2, LO5
9. What is a Chicago ward? LO1, LO5
10. How many wards are there in the City of Chicago? LO1, LO5
11. What does an alderman do? LO2, LO5
12. What is the City Council? LO1, LO5

Lorem Ipsum

Age Range Education
1. 21+ Master's Degree
2. 21+ Master's Degree
3. 21+ Master's Degree
4. 21+ Master's Degree
5. 21+ Bachelor's Degree
6. 21+ High School
7. 13+ High School
8. 13+ High School
9. 13+ High School
Data Collection

Lorem Ipsum

Data Analysis

Lorem Ipsum

Data

Participant Pretest: Correct Pretest: Score Posttest: Correct Posttest: Score Difference
1. 6 55% 8 73% 2
2. 8 73% 11 100% 3
3. 6 55% 9 82% 3
4. 4 36% 4 36% 0
5. 5 45% 10 91% 5
6. 9 82% 10 91% 1
7. 8 73% 11 100% 3

Analysis

Pretest

  • Average score: 60%
  • Average correct answers: 6.57
  • Mode: 6 and 8
  • Sample Standard Deviation: 1.81
  • Population Standard Deviation: 1.68

Posttest

  • Average score: 82%
  • Average correct answers: 9
  • Mode: 10 and 11
  • Sample Standard Deviation: 2.45
  • Population Standard Deviation: 2.27

Average of Standard Deviations

  • Sample Standard Deviation: (1.81+2.45)/2 = 2.13
  • Population Standard Deviation: (1.68+2.27)/2 = 1.97

Effect Size Calculation

Step 1: Difference between average correct answers

  • The posttest average (9 correct) minus the pretest average (6.57 correct) yields a mean gain of 2.43 correct answers.
  • 9-6.57 = 2.43

Step 2: Divide by the pooled/average standard deviation

  • This gain is divided by the average sample standard deviation across pretest and posttest (2.13) to calculate Cohen's d, standardizing the gain relative to the spread of scores.
  • 2.43/2.13 = 1.14

T-Test Calculation

Step 1: Calculate the difference between posttest and pretest scores for each participant, then find the mean difference

  • Mean difference = 2.43
  • 2 + 3 + 3 + 0 + 5 + 1 + 3 = 17
  • 17/7 = 2.43

Step 2: Calculate the standard deviation of the per-participant differences from Step 1

  • SD of differences = 1.62
  • Subtract the mean (2.43) from each difference (2, 3, 3, 0, 5, 1, 3), then square it
  • Sum the squared deviations: 0.18 + 0.33 + 0.33 + 5.90 + 6.60 + 2.04 + 0.33 = 15.71
  • Divide by n-1: 15.71/6 = 2.62
  • Take the square root: √2.62 = 1.62

Step 3: Calculate t

  • t = 3.97
  • 2.43/(1.62/√7)
  • 2.43/0.612 = 3.97

Step 4: Degrees of freedom

  • n-1 = 6

Step 5: Determine p-value

  • p = 0.007

Effect Size

The effect size is 1.14 > 0.8.

Results

A Cohen's d of 1.14 indicates a large effect size (d > 0.8), meaning the improvement from pretest to posttest represents a substantial change, not just a statiscally detectable one. Combined with the paired t-test results (t(6) = 3.97, p = 0.007), this provides strong evidence that the training had a real and sizeable impact on participant performance.

Findings and Opportunities

Lorem Ipsum


AI disclaimer: This case study uses Claude.ai to proofread copy, validate the data analysis, and collaborate to break down the calculation processes.